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A3STRA.CT 

In  this  report  a  class  of  linear  quadratic  pursuit -evasion  games  with 
bounds  on  the  pursuer's  control  has  been  defined.  In  this  problem  the 
pursuer's  control  appears  linearly  In  the  Hamiltonian  and  singular  arcs 
may  occur  In  the  solution.  A  sufficient  condition  for  the  existence  of 
a  saddle  point  for  this  class  of  problems  Is  derived. 

An  Indirect  numerical  technique  has  been  proposed  to  generate  a  rapid 
and  accurate  solution  to  a  class  of  problems  with  linear  state  equations 
In  which  singular  arcs  occur.  Then,  by  linearization  this  technique  Is 
extended  to  solve  a  class  of  problems  with  non-llnear  state  equations. 

By  modifying  this  technique  a  second  approach  has  ’-een  obtained  that  can 
solve  a  broader  class  of  singular  problems  with  linear  state  equations. 


The  effect  of  the  deviation  of  one  player  from  the  saddle  point 
strategy  on  the  performance  Index  and  the  opponent's  strategy  lias  been 
studied  for  a  two  person  zero-sum  differential  game  with  perfect 
Information.  An  inverse  system  technique  is  used  to  determine  the 
opponent's  strategy  by  periodically  measuring  the  state  or  the  output 
of  the  system.  Then,  the  proposed  technique  for  singular  problems  is 
applied  periodically  to  generate  an  approximate  closed  loop  solution 
(which  takes  Into  consideration  the  deviation  of  the  opponent  from 
the  saddle  point  trajectory)  to  achieve  better  performance  than  simply 
following  the  optimal  open  loop  solution.  A  numerical  example  is 
presented  to  Illustrate  the  efficiency  of  the  proposed  algorithm, 
and,  comparisons  have  been  made  between  the  results  of  the  open  loop 
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ABSTRACT 


In  this  report  a  class  of  linear  quadratic  pursuit -evasion  games  with 
bounds  on  the  pursuer's  control  has  been  defined.  In  this  problem  the 
pursuer's  control  appears  linearly  in  the  Hamiltonian  and  singular  arcs  ' 
may  occur  in  the  solution.  A  sufficient  condition  for  the  existence  of 
a  saddle  point  for  this  class  of  problems  Is  derived. 

An  Indirect  numerical  technique  has  been  proposed  to  generate  a  rapid 
and  accurate  solution  to  a  class  of  problems  with  linear  state  equations 
in  which  singular  arcs  occur.  Then,  by  linearization  this  technique  Is 
extended  to  solve  a  class  of  problems  with  non-linear  state  equations. 

By  modifying  this  technique  a  second  approach  has  been  obtained  that  can 
solve  a  broader  class  of  singular  problems  with  linear  state  equations. 

The  effect  of  the  deviation  of  one  player  from  the  saddle  point 
strategy  on  the  performance  Index  and  the  opponent's  strategy  has  been 
studied  for  a  two  pers(»i  zero-sum  differential  game  with  perfect 
Information.  An  Inverse  system  technique  Is  used  to  determine  the 
opponent's  strategy  by  periodically  measuring  the  state  or  the  output 
of  the  system.  Then,  the  proposed  technique  for  singular  problems  Is 
applied  periodically  to  generate  an  approximate  closed  loop  solution 
(which  takes  Into  consideration  the  deviation  of  the  opponent  from 
the  saddle  point  trajectory)  to  achieve  better  performance  than  simply 
following  the  optimal  open  loop  solution.  A  numerical  example  is 
presented  to  Illustrate  the  efficiency  of  the  proposed  algorithm, 
and,  comparisons  have  been  made  between  the  results  of  the  open  loop 
and  closed  loop  solutions. 
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CHAPTER  1 

nnTBODUCTIOir.  REVIOf  AND  CIASSIFIGATION  OF  GAMES 
SCOfE  OF  THE  DISSESIATIOR 

Hi  the  literature  of  zero-sum  differential  game  theory  a  great 
deal  of  attention  has  been  directed  towards  the  pursuit-evasion  games. 
This  Is  due  to  applicability  of  this  theory  to  military  oriented  prob¬ 
lems  or  physical  problems  in  areas  su6h  as  engineering  and  economics. 

In  many  problems  In  these  areas  signularlty  Is  a  source  of  difficulty 
for  obtaining  an  exact  solution.  Singular  arcs  qyilte  often  occur  in 
problems  lAen  one  or  more  components  of  the  controls  appear  linearly 
In  the  Hamiltonian  function.  Some  effort  Is  recpilred  to  study  differ¬ 
ential  games  vltb  singular  arcs  and  to  find  an  efficient  technique  to 
generate  an  accurate  solution.  Analytical  solutions,  in  general,  can¬ 
not  be  found  except  for  linear  quadratic  and  a  few  simple  non-Unear 
differential  game  problems.  Therefore,  this  research  Is  focused  oi 
the  computatloaal  aspects  of  the  singular  problems.  Our  effort  Is  to 
find  an  efficient  technique  with  accuracy  emd  rapid  convergence  which 
can  be  used  for  m-llne  purposes. 

A  common  approach  to  the  zero-sum  differential  game  Is  to  find  a 
Joint  optimal  solution  for  which  each  player  assumes  the  other  player 
Is  rational  and  uses  his  optimal  strategy.  Although  this  assumption 
basically  is  correct  there  are  many  reasons  why  a  player  may  not  be 
la  to  fol^  wr  the  exact  saddle  point  solution.  For  example,  bias  error 
In  **’  eobCroUer,  a  lack  of  accuracy  in  the  computational  technique  or 


subpptinal  playing  of  players  will  cause  some  deviation  from  the  real 
optiinal  trajectories. 

One  way  that  a  player  can  take  partial  advantage  of  the  deviation 
of  the  opponent  from  the  optimal  open  loop  strategy  Is  to  measure  the 
state  of  the  game  periodically  and  at  eetch  time  of  measurement  apply 
the  saddle  point  strategy  assuming  the  other  player  plays  optimally. 

It  would  he  more  desirable  to  find  out  the  oppenent's  non>optimal 
strategy  (if  it  Is  possible)  and  consider  this  strategy  as  eui  cuidltlonal 
known  input  to  the  system  and  solve  a  one-sided  optimization  problem. 

By  this  method  one  can  gain  more  than  he  could  by  using  his  own  optimal 
open  loop  policy. 

In  differential  games  the  strategy  of  the  opponent  can  be  deter¬ 
mined  periodically  throu^  an  inverse  system  if  such  a  system  exists 
and  there  is  perfect  information  of  the  system  states.  By  periodically 
solving  an  optimal  control  problem,  using  the  proposed  numerical  tech¬ 
nique  an  approximate  closed-loop  strategy  dependent  solution  can  be 
obtained  for  singular  problems. 

:Di  order  to  put  the  present  work  in  perspective  we  briefly  review 
the  highli^ts  of  the  game  theory  in  this  chapter. 

1.1  Review  and  Classification  of  Games 

In  the  game  theory  there  are  different  classification  schemes. 

One  of  the  proposed  shcemes  is  as  such: 

Static  Games,  which  are  not  associated  with  time,  and 

Dynamic  Games,  which  have  time  evolution.  All  kinds  of  game  problems 

are  based  on  the  r  sicept  of  optimization  cheory. 
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Both  static  and  dynamic  optimization  problems  consist  of  three  des¬ 
criptive  elements  which  were  defined  by  Ho^^^  in  the  framework  of 
Generalized  Control  Theory.  These  are  payoff  functions  or  performance 
index,  controller,  and  avalalable  data  to  the  coitroller.  Once  these 
elements  are  specified,  further  classifications  can  be  made. 

Static  games  can  be  continuous  or  discrete.  In  a  continuous  static 
game,  the  payoff  function  Is  expressed  as  a  continuous  algebraic  rela¬ 
tionship  and  there  are  Infinite  numbers  of  strategies  for  each  player. 
The  discrete  static  game  Is  sometimes  called  a  matrix  or  bimatrix  game. 
In  these  types  of  games  there  are'  finite  numbers  of  strategies  for  each 
player.  If  the  sum  of  the  payoffs  to  each  of  the  players  Is  zero,  the 
game  Is  a  zero  sum  game,  otherwise  it  la  a  non -zero-sum  game.  If  the 
sum  of  the  payoffs  Is  constant  the  game  Is  called  a  constant  sum  game. 

The  other  major  class  of  games  is  dynamic  or  differential  games 
in  which  the  state  of  the  game  evolves  as  a  continuous  function  of 
time,  and  the  strategies  in  these  games  are  also  continuous  functlcxis 
of  time.  In  this  type  of  game  the  system  Is  generally  expressed  by  a 
vector  differential  equation  of  the  form; 

x(t)  «  f(x(t),u(t),v(t),t) 

with  a  given  initial,  state  xit^)  »  x^  and  a  scalar  payoff  function 

J  -  h(x(t^J^tp  +  j  L(x(t),uCt),v(t),t)dt 
^0 

when  the  state  'of  the  game  evolves  in  discrete  times  and  strategies  are 
also  implemented  In  discrete  time,  then,  the  game  is  called  a  discrete 
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differential  game.  In  this  type  of  ^zne  the  dynamic  system  Is  genereLUy 
expressed  hy  a  vector  difference  equation  of  llie  form: 

Vi  ■ 

vlth  a  given  initial  state  x(0)  s  and  a  scalar  payoff  function 

N-l 

&:0. 

Games  can  he  categorized  aiccordlng  to  the  information  data  to  the 
controllers.  When  each  player  has  perfect  information  ahout  the  state 
of  the  game,  the  system  structure  eind  the  other  player's  payoff 
funetlon,  then,  the  game  Is  called  a  perfect  information  game.  If  some 
of  the  states  are  not  measurahle  In  the  game  and/or  parameters  of  the 
game  are  tinknown  (at  least  to  one  player)  an  imperfect  Information 
game  results.  The  game  Is  c€J.led  stochastic  game  If  at  least  one 
player  has  uncertain  or  randcm  Imowledge  of  scoie  states  of  system's 
parameters . 

Games  can  be  classified  according  to  the  goal  of  the  game. 

Game  of  Kind  or  Qualitative  Game,  ia  this  game  the  payoff  func¬ 
tion  of  the  game  is  usually  expressed  in  a  discrete  way  as  a  win  or 
loss.  For  example,  the  caputre  occurs  or  does  not  occur  in  a  pursuit- 
evasion  game. 


Game  of  Degree  or  Quantitative  Game .  In  this  type  of  game  the 


payoff  is  usually  a  continuous  functional.  For  example.  In  a  pursuit - 
evasion  game,  one  player  tries  to  minimize  and  the  other  tries  to 
maximize  the  separation  distance  at  some  final  time.  It  Is  assumed 
that  a  capture  does  not  occur  during  the  game. 

Strategies  which  are  used  in  dynamic  games  can  he  classified  as: 

1.  Open  Loop  Strategies,  In  ^ich  controls  are  functions  of 
time  and  Initial  conditions,  e.g.,  u  ■  u(t,XQ,tQ).  The  dlsadvantaige 
of  this  kind  of  strategy  In  dynamic  game  problems  Is  the  inability  to 
take  advantage  of  non-optlmally  playing  of  the  opponen^s.  This 
strategy  can  be  Identified  for  the  entire  time  of  the  game  before  the 
game  starts. 

2,  Closed  Loop  Strategies,  In  which  ccxitrols  are  functioos  of 
time  and/or  current  state,  .e.g.,  u  «  u(x,t).  This  kind  of  strategy 
can  take  partial  advantage  of  ncxi -optimally  playing  of  the  opponent, 
because  it  vlU  be  assumed  that  the  opponent  will  play  optimally  frcm 
the  time  of  measurement  on. 

In  a  differential  game  problem  if  a  player  chooses  his  strategy 
based  on  Information  of  his  opponent  non-optlmally  playing  strategy, 
he  may  be  able  to  achieve  a  better  performance  them  the  closed  loop 
solution.  The  strategy  which  Is  chosen  in  this  meuoner  Is  celled 
closed-loop  strategy  dependent. 

Pure  and  Mixed  Strategies.  If  at  each  Instant  of  time  during  the 
evolution  of  the  gome  the  veklues  of  the  strategies  eire  known  the  game 
Is  called  pure  strategy  game.  But  if  the  strategies  at  each  point  of 
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time  are  randomized  and  their  characteristics  are  specified  hy  prohabll- 
Ity  density  fhnctlons,  the  game  Is  called  mezed  strategy  game.  Pore 
strategy  games  can  he  considered  as  a  special  case  of  mixed  strategy 
e^unes  In  which  the  proibahillty  of  some  particular  controls  are  equal  to 
one.  Altiiouj^  in  a  static  or  a  discrete  and  multistage  game  the  con¬ 
cept  of  mixed  strategy  can  he  easily  Tisuallzed,  In  continuous  dynamic 
games  the  vlsuallzatloa  is  very  difficult.  ThuB,  it  has  not  found 
much  application  in  the  analysis  of  differential  games.  In  dynamic 
games  pure  or  mixed  strategies  can  he  open  loop  or  closed  loop. 

1.2  Types  of  Solutions 

As  was  mentioned  earlier  in  this  chapter,  games  may  he  classified 
as  zero-sum  or  non-zero  sum  games. 

^  zero  sum  geoses  one  player  tries  to  minimize  and  the  other  player 
tries  to  maximize  payoff  functicms.  In  such  games  a  saddle  point  solu¬ 
tion  is  sought  such  that  each  player  optimizes  his  ohjective  assuming 
the  other  player  does  the  same.  The  resulting  value  of  the  payoff 
functioi  is  called  the  value  of  the  game.  The  mathematical  definition 
of  the  saddle  point  for  both  static  and  dynamic  games  is: 

J(vi*,v)  <  J(u*,v*)  <  J(u,v*) 

where  in  a  dynamic  game  u  and  v  cure  functions  of  time  cud/or  curz«nt 

states. 

In  non-zero-sum  games  there  are  different  types  of  solutions,  l.e., 
Nash  equilibrium  solutions,  non-inferior  solution  (or  pcurato  optimal 
sets)  and  minmax  solutions. 
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Nash  Solatlons.  If  Is  the  strategy  for  1^^  player  and 
is  his  cost  fonctlonal,  then  a  Nash  solution  is 
defined  by  the  foUoiring  relatimship: 

^  ^  #  # 

^  J *  *  *  ’'^i*  *  *  * 

vhere 

^  u^  i  B  ly  •  •  •  jN 

which  implies  that  no  player  can  improve  his  payoff  by  unilaterally 
deviating  from  his  Nash  solution,  provided  all  other  players  use  their 
Nash  strategies.  This  solutiai  has  the  characteristic  of  being  pro¬ 
tected  against  cheating;  however,  each  player  should  play  ratlonsiUy, 
i.e.,  no  player  can  attend  to  increase  another  player's  cost  without 
regard  to  his  own  loss. 

Note:  When  the  gome  is  zero  sum,  i.e.,  »  -Jg  the  Nash  solu¬ 

tion  is  the  same  as  the  saddle  point  solution. 

Noninferior  Solutions.  It  may  be  possible  to  achieve  simultan¬ 
eously  a  superior  payoff  than  the  Nash  solution  which  was  a  non-cooper¬ 
ative  solution,  if  certain  cooperation  among  the  players  is  made  in  a 
prescribed  manner. 

This  kind  of  solution  is  used  in  classical  game  theory  modem 
welfare  econoailcs.  ^  this  set  of  solutions  no  player  M-n  achieve  a 
better  pay-off  unless  it  is  at  the  expense  of  the  other  player. 


Mln4iax  Solatlon.  This  Idnd  of  solutlcxi  represents  a  "security 

^  lih 

level"  for  each  pla3rer.  The  mln^iax  soloitioa  for  the  1  player 
Is  the  strategy  which  satisfies 

JjCu.)  ■  min  max  J.(u_ V  j^i 
11  u-  u_  ^  ^ 

i  u 

This  relationship  implies  that  the  other  players  try  to  do  the 
th  th 

worst  damage  to  the  i  player  -and  the  i  player  tries  to  gain 
the  most. 

1.3  Review  of  Methods  of  Soluti<ms  for  Differential  Games 

The  study  of  differential  games  was  initiated  by  Isaacs  in  19?^. 

His  approach  was  formal  and  closely  resembled  the  dynamic  programming 
approach  to  optimization  problems.  In  196^  Berkowitz  and  Fleming 
applied  a  calculus  of  variation  technique  to  a  simple  differential 
game.  later  on  Berkowltz  treated  a  wider  class  of  differential  games 
by  the  same  technique.  More  recently  functional  analysis  has  been 
applied  to  differential  game  problems  as  a  rigorous  approach,  and 
certain  hi^ily  mathematicsl  problems  without  direct  physical  inter- 
pretatlon  have  been  solved  by  this  approach  (Freedman^  *), 

Geometric  approach  is  another  rigorous  and  interesting  technique  to 
differential  games  which  provides  some  insight  into  the  problems  and 
has  been  used  by  some  authors  (Balaquler,  Gerald,  Lletman^^^). 

La  1969  Bryson  and  Ho^^^  treated  a  class  of  zero-sum-linear 
quadratic  differential  games  and  by  the  application  of  the  set  of 
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necessary  conditions  for  the  saddle  point.  They  obtained  a  set  of 
closed  loop  solutions.  Then  by  forming  some  auxiliary  problems  they 
verified  the  existence  of  the  saddle  point.  In  1770,  McFarland  used 
the  same  class  of  problems  and,  without  the  assumption  of  a  saddle 
point  used  another  approach  to  show  the  existence  of  the  saddle  point. 
Althou^  the  obtained  solutions  for  this  class  of  problems  are  euialytic- 
al,  however,  the  solution  to  the  same  class  of  problems  with  control 
and/or  state  constraints  generally  cannot  be  analytlcal^^'^^. 

In  order  to  solve  problems  not  having  analytical  solutions,  some 
numerical  techniques  have  been  proposed  to  generate  optimal  open  loop 
solutions.  In  these  techniques  It  hsis  been  assumed  that  the  saddle 
point  solution  exists  and  singularity  does  not  occur.  Sevdral  tech¬ 
niques  which  are  used  In  optimal  control  problems  have  been  applied  to 
differential  game  problems.  Including  nei^boring  optimal  techniques 
(which  are  closely  associated  with  successive  sweep  method) ,  quasl- 
llnearization,  and  differential  dynamic  programming. 

There  are  a  few  closed  loop  techniques  that  have  been  proposed 

fhO 

for  generating  a  near  optimal  solution.  Anderson  ’  ^s  worked  on 
8U1  updating  technique  for  generating  a  near  optimal  closed  loop  solu¬ 
tion  to  a  zero  sum  perfect  information  differential  game  by  period¬ 
ically  updating  the  solution  to  the  two  point  boundary  value  problem 

obtedned  by  the  application  of  the  necessaury  condition  for  the  saddle 

(hi) 

point  solution.  Jachimowltz  has  proposed  an  adaptive  technique, 
based  on  estimation  theory,  to  determine  the  non-optimal  strategy  of 
the  opponent  throuc^  'the  state  measurement.  Then  by  converting  the 
game  to  a  one-sided  optimal  control  problem  he  generated  a  near  optimsLl 
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closed  loop  solatlon  for  non-singular  differential  gaaes.  Bebn  and 
and  Gmzalez^^^  have  used  Inverse  systems  to  determine  the 
opponent  strategy  for  the  solution  of  non-singular  stodiastlc  and 
deterministic  zero  stim  differential  games. 

The  review;  of  the  literature  in  this  aurea  shows  that  the  compata- 
tlonal  aspect  of  singular  game  problems  requires  special  attentlm  and 
it  Is  the  subject  of  this  dissertation. 

l.U  Objective  and  Scope  of  the  Dissertation 

It  is  the  main  purpose  of  this  dissertation  to  present  a  tech¬ 
nique  that  generates  an  accurate  and  rapid  optimal  open  loop  solution 
to  a  class  of  singular  differential  game  problems. 

In  the  next  chapter  some  definitions  and  theorems  which  sure  common 
between  sigoular  optimal  ccntrol  theory  and  singular  differential  game 
theory  have  been  introduced.  These  results  are  very  useful  in  provid¬ 
ing  insight  to  the  form  of  the  singular  solutions  of  the  problems.  In 
this  chapter  we  define  a  new  class  of  two-person  zero-sum  differential 
games  (genersdized  pursuit -evasion  game)  with  pure  strategy  and  per¬ 
fect  information,  which  have  linear  state  equations.  In  this  problem 
the  pursuer's  and  evader's  controls  are  respectively  bounded  and  un¬ 
bounded  and  the  performance  index  is  quadratic  in  terms  of  the  state 
and  the  evader's  control.  The  flnail  time  is  fixed  and  the  game  is 
considered  as  a  game  of  degree. 

Some  conditions  for  the  strict  convexity  and  strict  concavity 
of  the  performance  izidex  with  respect  to  the  pursuer's  and  the  evader's 
controls  are  derived  to  guarantee  the  existence  of  a  unique  saddle 


point  solation  in  this  class  of  probleias 


In  Chapter  3  an  Indirect  numerical  technique  vlth  tvo  approaches 
are  offered.  The  first  approach  can  only  solve  problems  with  singular 
arcs.  In  this  technique  the  sequence  of  controls  In  the  entire  time 
Interval  of  the  game  are  estimated.  The  solution  to  the  set  of  two 
point  boundary  value  problems  (obtained  from  the  necesseury  condlt lexis 
for  optimality)  Is  generated  by  an  Iterative  procedure  using  Newton's 
method.  This  approach  iterates  only  on  switching  times  between  control 
arcs  and  has  at  least  quadratic  convergence.  This  technique  Is  extended 
to  a  class  of  non-linear  differential  games  by  linearization. 

The  second  approach  which  is  somehow  similar  to  the  first  approach 
ntm  also  solve  linear  bang  band  and  highly  dimensional  problems.  The 
solution  to  the  set  of  TPBVP  Is  obtained  by  Iterating  on  the  initial 
costates  and  switching  times  between  control  arcs. 

Numerical  results  of  a  physical  example  are  reported  In  this 
chapter. 

Chapter  Iv  discusses  closed  loop  on  line  solutions  for  this  class 
of  differential  games.  The  concept  of  the  inverse  system  Is  introduced. 
The  existence  of  the  Inverse  system  Is  discussed  and  an  algorithm  Is 
used  which  incorporates  the  necessary  and  sufficient  condition  for  its 
existence.  The  proposed  algorithm,  together  with  the  inverse  system  is 
applied  to  generate  an  approximate  closed-loop  strategy  dependent  solu¬ 
tion.  Computational  comparisons  have  been  made  between  the  saddle  point 
solutions  and  the  cases  that  one  player  deviates  from  the  saddle  point 
strategy,  and  the  other  plays  open  loop  and  closed  loop  strategies. 


U 


Chapter  5  eumaarlzes  adl  the  results  obtained  in  this  report, 
AdTaatages  and  disadvantages  of  numerical  techniques  are  discussed. 
Sane  areas  of  work  for  future  research  are  also  reccmnended. 


CmPFER  2 


SmGUIAR  OPTIMAL  CCNTROL  AND  DIFFERENTIAL  GAME  FROBI01S 

Singular  arcs  may  arise  In  many  optimal  control  problems.  In 
the  past  two  dejeades  singularity  has  received  considerable  theoretical 
attention.  This  problem  was  defined  by  Rosonoer  (1939)  and  has  been 
studied  by  Johnson  and  Gibson  (I963),  Robbins  (1966),  Goh  (I966), 
McDonell  and  Powers  (I97O).  The  authors  have  developed  sooie  necessary 
conditions  and  also  Md}onell  and  Powers  have  obtained  sufficient 
conditions  for  optimal  control  assuming  there  exists  a  totally  sing¬ 
ular  extremal.  Singular  solutions  in  optimal  control  have  been  thought 
by  many  people  to  be  of  only  academic  interest.  However,  singular 
arcs  quite  often  appear  In  engineering,  economics  and  chemical  problems. 
Slebenthal  and  Arls  (1961^)  have  shown  that  optimal  singular  arcs  occur 
in  chemical  reactor  stcurtup  problems.  Optimal  trajectories  of  mass- 
varying  vehicles  which  are  subjected  to  Mrodynamlc  forces  include 

singular  arcs.  The  sounding  rocket  problem^  %  Saturn  Guidance  singulskr 
(22) 

flat  earth'  *  and  resource  allocation  problems  are  other  examples  of 
singularities.  In  the  following  section  we  will  formulate  an  optimal 
control  problem  and  discuss  the  possibility  of  occurrence  of  singular 
arcs  and  later  on  we  will  extend  this  subject  to  differential  game 
problems . 

2.1  Problem  Formulation 

The  fundamental  problem  of  optimal  control  theory  can  be  formu- 

f 

lated  in  equivalent  foms  of  Bolza,  Mayer  and  Leigrange.  The  Bolza 
problem  is  the  following: 
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Find  the  control  flinctlon  u(*)  which  minimizes  (or  maximizes) 
the  performance  functional 

J  -  h(x(t^),tp  +  J  L(x,u,t)dt  (2.1-1) 

■*^0 

subject  to: 

x(t)  -  f(x(t),u(t),t)  (2.1-2) 

x(tQ)  s  Xq  given  (2.1-3) 

u(*)  belongs  to  the  set  d.  and  t  is  a  member  of 

X  is  an  n  dimensional  state  vector,  u  is  an  m  dimensional  control 
vector,  h  and  L  are  scalar  functions  and  are  assumed  smooth.  The 
control  set  is  defined  by 

U  s  fn(  •)!'»ij^(  •)  is  piecewise  continuous  in  time,  |uj^(*)i 

tQ  ^  t  <  tj,,  1  *  1,2, ...,m}  (2.1-i^) 

The  initial  state  and  Initial  time  are  specified  in  the  final 
state  and  final  time  may  be  specified  or  unspecified. 

As  a  prerequisite  to  solving  this  problem  by  the  maximum  principle 
we  define  the  Hamiltonian  as 

H(x,u,\,t)  s  L(x,u,t)  +  \^f(x,u,t) 

* 

Necessary  conditions  for  u  to  be  an  optimal  control 
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(2.1-5) 


±*(t)  »  ^  (x*(t),u*(t),t) 

(2.1-6) 

\*(t)  «  ^  (x*(t),u*(t),\*(t),t) 

(2.1-7) 

H(x* (t)  ,u*(t)  ,\*(t)  ,t)  ^  H(x* (t)  ,u(t)  ,x.* (t)  ,t) 

(2.1-8) 

for  all  admissible  u(t)  and  for  all  t  e  and  boundary  con¬ 

ditions 

[  ^  (x*(tp,tp  -  5X^  +  [H(x*(tp,u*(tpA*(tf),tp 

+  ^(x(tp,tp]6t^-  0  (2.1-9) 

where  x.  is  an  n  dimensional  lagrangian  multiplier  vector.  The 
optimal  control  n  should  satisfy  (2.1-8)  and  usually  extremal  u  is 
obtained  as 


u*  -  arg  min  H(x,u,\,t)  (2.1-10) 

u 

and  in  the  case  that  there  are  bounds  on  the  control  such  that 

|uj^(t)|  SK^(t)  i-l,2,...,m  (2.1-11) 

and  the  Hamiltonian  is  linear  in  terms  of  u^  assuming  the  components 
of  the  control  are  Independent,  the  extremal  controls  can  be  expressed 
as 
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-  K^{t)aga.  if  HUj^(t)  ^  0  for  t  e  (2.1-12) 


where  belongs  to  time  Interval  [tQ,t^] 

+1  If  aa.  <  0 

sgn  Hn.  -  ;  ^  (2.1-13) 

;  ^  -1  if  Hn.  >0 

X 

Note  that  there  may  exist  conditions  such  that  ■  0  for  some  non¬ 
zero  time  Interval.  Then  we  will  have  problems  of  singularity.  With 
respect  to  this  more  general  problem  some  basic  definitions  and  theorems 
of  singular  optimal  control  problems  are  Introduced. 

Definition  2.1.  If  one  or  more  components  of  the  control  function  u 
appear  linearly  in  the  Hamiltonian,  and  there  exists  a  non -zero  time 
Interval  [tj^jtg]  in  [tQ,t^]  such  that  the  coefficient  of  at  least 
one  of  these  components  are  zero  on  [t^^jtg].  Then  the  coatrol  is  said 
to  be  singular.  In  this  Intez^ral  maximum  principle  (2.1-6)  provides  no 
information  about  the  control  u  and  its  relationship  with  state  and 
costate  X  and  \  . 

Definition  2.2.  Let  u^^  be  the  ith  element  of  the  optimal  singular 
control  vector  u  on  the  interval  belonging  to  [t^jt^.], 

^Ich  appears  linearly  in  the  Hamiltwiian .  Let  2q  be  the  lowest 
order  of  the  time  derivative  of  Hu^  in  which  u  appears  explicitly 
with  a  coefficient  which  is  not  identically  zero  on  the  sub  interval  of 
^^1*^2^*  called  the  order  of  the  singular  subeurc. 
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Definition  2.3.  Assuming  all  the  components  u.  .u_ . u  of  the  con-r 

12  m 


trol  vector  u  are  singular  simultaneously,  then  u  is  called  a 
totally  singular  control  function  when 


55"  (x,\,t)  «  0  for  t  e  [^Qft^] 


(2.1-11^) 


Definition  2.4.  If  (2.1-llv)  holds  for  arcs  in  K  subintervals  of 
length  T^,  1  s  1,2, ...,K  such  that 


K 

2  T 

1.1 


i 


^f  ■ 


(2.1-15) 


then  the  problem  is  called  partially  singular. 

So,  for  the  existence  of  the  singular  arc  it  is  necessary  that  the 

Hamiltonian  be  a  linear  function  of  at  least  one  eomponent  of  a  control 
vector.  The  analysis  of  such  problems  la  canplicated  by  the  fact  that 
the  solution  in  general  consists  of  seme  ccmbinatlcxi  of  singular  and 
non-singular  subarcs.  The  number  and  sequence  of  these  subarcs  are  not 
Imown  a  priori,  and  it  is  almost  impossible  to  establish  the  existence 
of  singular  arcs  without  actual  numerical  solutions. 

The  following  theorem  which  is  a  necessary  condition  for  the  opt¬ 
imality  of  singular  subarcs  is  due  to  Robbins and  junction  theorems 
which  are  followed  are  given  by  McDonell  and  Powers  ^  . 

Theorem  2.1  (Generalized  Legendre -Clebach  Condition) .  On  an  optimal 
singular  subarc  of  order  q,  it  is  necessary  that 


(-1) 


a  0 


(2.1-16) 
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and  if  (»ly  Inequality  holds  it  is  called  strengthened  GLC. 

The  essence  of  the  proof  is  given  in  Appendix  A  and  the  complete 
proof  is  found  in  Reference  (3I). 

Since  d^!  /  dt  ^  Hu  is  the  lowest  time  derivative  of  Hu  in 
which  control  u  appears  explicitly  in  the  general  form,  we  can  have 

.2 

— ^Hu(x,\,t)  «  A(x(t),>,(t),t)  +  B(x(t),\(t),t)ug  (2.1-17) 

dt 

A  and  B  as  a  function  of  time  are  defined  as 

a(t)  2  A(x(t),\(t),t)  (2.1-li 

B(t)  2  B(x(t)A(t),t)  (2.1-19) 

The  above  notations  are  used  in  the  proof  of  the  theorems  in  this 
(;hapter. 

2.2  The  Junctions  Theorems.  Although  the  analysis  of  totally  singular 

control,  problems  are  rather  well  developed,  in  partially  singular 

coatrol  problems  the  analysis  of  junction  points  are  not  yet  fully 
% 

understood.  Since  a  useful  sufficient  condition  for  such  problems  is 
not  available,  one  has  to  study  the  necessary  conditions  which  are 
valid  in  the  neighborhood  of  a  junction  between  singular  stud  nor- 
slngular  subarcs.  It  is  expected  that  such  conditions  can  b?  used  to 
eliminate  candidate  extremals  or  predict  beforehand  the  way  in  which 
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singular  and  non-singular  subarcs  must  be  Joined  or  whether  the  optimal 
control  is  continuous  or  discontinuous  at  a  Junction  point. 

Assuming  the  optimal  control  is  well-behaved  in  a  neighborhood  of 
a  Junction,  then  the  following  theorem  holds. 

Theorem  2.2.  Let  t_  be  a  point  at  which  slnguleu*  and  non-singular 

arcs  of  an  optimal  control  u  are  Joined,  and  let  q  be  the  order  of 

singular  subarcs.  Suppose  the  strengthened  GLC  condition  is  satisfied, 

and  assume  that  the  control  is  piecewise  analytic  in  a  neighborhood  of 

Ir) 

t  .  Let  u'  (r  2  O)  be  the  lowest  order  time  derivative  of  u 
s 

which  is  discontinuous  at  t  .  Then  q  +  r  is  an  odd  integer  (proof 

s 

la  given  in  Appendix  A) . 

Two  corollaries  follow  from  Theorem  2.2. 

Corollary  1.  In  <1  even  problems,  assuming  u  is  piecewise  analytic, 
and  the  strengthened  GIC  condition  is  satisfied,  then  the  optimal  con¬ 
trol  is  continuous  at  each  Junction. 

Corollary  2.  In  q  odd  problems,  assuming  u  is  piecewise  analytic, 
and  the  strengthened  GIC  condition  is  satisfied,  then  the  optimal  con¬ 
trol  either  has  a  Jump  discontinuity  at  each  Junction  or  else  the 
singular  control  Joins  the  boundary  smoothly,  i.e.  with  a  continuous 
first  derivative. 

For  Theorem  2.2  strengthened  GLC  conditions  should  be  satisfied 

at  the  Junction  point  t  .  There  is  also  a  possibility  that  the  GLC 

6 

condition  be  satisfied  with  equality  at  this  point.  If  q  is  the 
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order  of  the  sijtgular  arc  •  P(t)  cannot  be  identically  zero 

(»  the  singular  subarc.  Therefore,  in  view  of  analytlclty  assumptions, 
a  derivative  of  some  order  of  3(t)  must  be  non-zero  at  time  t 

s 

even  if  P(t  )  ■  0.  This  leads  to  the  generalization  of  the  Theorem  2.2 
0 

which  is  stated  as  a  sepeirate  theorem  to  emphasize  the  Important  results 
of  Theorem  2.2. 

Theorem  2.3.  Let  t_  be  a  point  at  which  singular  and  non-singular 

subarcs  of  an  optimal  control  u  .  are  Joined,  and  let  q.  be  the  order 

of  singular  arc.  Assume  that  the  control  is  piecewise  analytic  in  a 

neighborhood  of  t  ,  and  let  '  (m  ^  0)  be  the  lowest  order 

8 

derivative  of  the  GLC  expression  s  3  ^ich  is  non-zero  at 

t  ,  then 
s’ 

1.  if  m  ^  r,  q  r  -f  m  is  an  odd  Integer 

2.  if  m>r,  -sgn[P^“^(t^)p“(0]  -  (-l)'^'^^'^ 

The  proof  of  this  theorem  is  similar  to  that  for  Theorem  2.2. 

The  conclusion  of  this  theorem  is  that  if  m  >  r,  3^®^  may  not 
be  continuous  at  Junction  t^  and  if  m  ^  r,  is  continuous  at 

Junction  point  "tg*  .. 

Theorems  2.2  and  2.3  require  the  assumption  of  piecewise  analy- 
ticlty  of  the  control  in  a  neighborhood  of  the  Junction. 

This  hypothesis  is  usually  satisfied  on  the  singular  subarc,  but 
not  always  m  the  ncnslngular  subarc.  Thus,  we  are  led  to  consider 
properties  which  do  not  require  the  assumption  of  analytlclty  as  stated 
in  the  following  theorem. 
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Theorem  2,k,  Let  u  be  an  optimal  control  which  contains  both  ncn- 


slngular  subeurcs  and  piecewise  continuous  order  singular  subarcs. 

1.  If  Hu^  ^  0  on  the  non-singular  side  of  a  JunctlODy  then 
the  control  is  discontinuous. 

2.  If  A  s  0,  B  0  and  E  /  0  at  a  {Junction  then,  the  control 
is  discontinuous. 

3.  If  u  is  piecewise  continuous  on  the  non -singular  subarc 

d^‘  /  dt^  Hu  «  0  on  the  non-singular  side  of  a  jTonction, 
and  B  0  at  the  {Junction  then  the  control  is  continuous. 

Proof.  For  Case  l)  knowing  that  d^,  /  dt^  Hu  s  O  on  the  singular 
subarc  and  d^*^.  /  dt^  Hu  0  on  the  side  of  noasingular  subarc  we 
have 

“C^g)  t  P(ta)K(tg)  /  0  -  a(tg)  +  3(tg)ug(t^) 

From  this  relationship  we  obtain  /  K(t^).  Therefore  u 

S  3  S 

is  discontinuous. 

For  Case  2)  a(t  )  »  0  and  3(t„)  ^  0  imply  u  (t  )  «  0,  and 

S  S  S3 

since  K(t  )  /  0,  the  control  is  discontinuous. 

8 

For  Case  3)  since  d^  /  dt^*^  Hu  «  0  for  both  singular  and 
non -singular  subarcs,  we  will  have 

a(tg)  +  3(tg)u^(tg)  -  0  -  a(tg)  + 

and  since  P(tg)  0,  then,  Therefore  the  control 

is  continuous  at  the  {Junction. 
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order  to  see  hov  the  singular  subarc  may  occur  in  a  problem  ve 
consider  a  simple  scalar  exasqple. 

2.3  Example  of  a  Singular  C«itrol  Rroblem  .  Find  control  u  to 
minimize 

J  -  J  J  x^(t)dt  (2.3-1) 

0 


subject  to 


given 


i(t)  •  u(t)  i(t(j)  -  ijj 


|u|  SI 

j 

j 

'  !  vhere  t^  is  fixed  final  time. 

;  ,  The  Hamiltonian  is  defined 

I 

H  »  I  x^  +  \u 


(2.3-2) 


(2.3-3) 


(2.3-1+) 


the  set  of  necessary  and  boundary  conditions  are 

X  ■  u 

*  * 

\  (t-)  ■  0 


(2.3-5) 

(2.3-6) 


x(0)  -  X-. 


(2.3-7) 


The  optimal  control  u  Is  obtained  as  the  following 

[  +1  if  \(t)  <  0 

u  -  I  -1  if  \(t)  >  0 

I  \mdetermlned  If  \(t)  «  0 

If  \(t)  s  0  we  will  have 

X*(t)  -  0  ■>x(t)  «  0  «>x(t)  s  0  ■>u  (t)  -  0  (2.3-9) 

s 

Also  the  strengthened  GLC  condition 

(-l)^gu-l>0  (2.3-10) 

for  singular  arcs  is  sat sifted* 

Now  hy  changing  the  values  of  initial  condition,  final  time  and 
final  state,  we  will  consider  several  different  cases. 

Case  1.  Let  x(0)  >1,  m  1  and  x(l)  be  free,  then  the  solution 
will  be 

u*(t)  ■  -1 

x*(t)  -  -t  +  1  f  or  t  e  [0,1]  (2.3-11) 

X.  (t)  ■  -  t^  +  t  +  ^ 

The  optimal  control  and  trajectories  are  shown  in  Figures  (2.1-a) 
and  (2.1.b). 

•  • 
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We  see  that  with  this  Initial  state  and  final  time  slngulaxlty 


does  not  occur  in  any  InterTal  of  the  problem. 

Case  2.  Let  x(0)  ■  1,  •  2  and  z(2)  be  free,  then  the  solation 

is  obtckined  as 


and 


u  (t) 
x*(t) 
\(t) 


-1 


-t  +  1 


^  t  +  ^ 

2^  -t  +5 


for  t  e  [0,1] 


(2.3-12) 


u  (t)  *  0  for  1  <  t  <  2 


*/  V 

X  (t) 

\  (t) 


0  ) 


for  t  e  [1,2] 


(2.3-13) 

(2.3-11^) 


The  optimal  control  and  trajectories  are  shown  in  Figure  2.2-a 
and  2.2-b. 


Figure  2.2-a.  .State  and  costate  trajectories  in  nonsingular  and  singular 
intervals. 
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Figure  2.2-b.  Noosingular  and  singular  controls « 

It  Is  noticed  that  when  the  final  time  In  Case  1  Is  Increased  frcci  1 
to  2,  and  initial  state  Is  the  same  and  final  state  remains  trw, 
singular  arc  occurs  in  the  interval  t  e  [1,2]. 

Case  3«  let  z(0)  ■  1,  t^  ■  2  and  z(2)  ■  1.  The  solution  will  be 


Figure  2,3-l>.  Bang-Bang  Control. 
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I 


I 


i 


I 


I 


5 


In  this  case  we  have  fixed  the  final  state.  This  causes  the 
solution  of  Case  2  to  change  frcei  a  singular  solution  to  a  bang-bang 
solution. 


Case  U.  Now  we  modify  Case  3  by  increasing  the  final  time  frcm  t^  s  2 


to  tj  •  3* 

The  solution  is  obtained  as 


u*(t) 

**(t) 

X.*(t) 


-1 


-t  +  1 

2^  -  t  +5 


t  6  [0,1] 


(2.3-18) 


u(t)«0  0<t<2 


(2.3-19) 


x*(t)  -  0  I 
X*(t)  .  0  j 


t  e  [1,2] 


(2.3-20) 


At) 

x*(t) 

\{t) 


t  -  2 


1  2 

-  I  t*^  +  2t  -  2 


t  e  [2,3] 


(2.3-21) 


The  optimal  control  and  trajectories  are  shown  in  Figures 
2.4-a  and  2.4-b. 
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It  is  noticed  that  in  Case  3,  l3y  changing  the  final  time,  the  solu¬ 
tion  of  the  problem  changes  from  bang-bcmg  to  a  solution  with  singular 
arc.  Indeed  there  are  many  simple  changes  which  can  be  made  to  the 
effect  that  singularity  will  occur  or  be  remoyed.  For  example,  in 
Case  2,  if  the  initial  condltlcxi  is  increased  from  x(0)  si  to  x(0)  »  2 
with  the  same  time  duration,  singular  control  will  be  removed  from  the 
solution  and  the  control  solution  will  stay  on  the  lower  bound  until  the 
final  time  is  reached. 

Singular  arcs  have  also  appeared  in  differential  game  problems. 

The  Homicidal  Chauffeur  game^^^  is  a  pursuit-evasion  game  with  the 
possibility  of  singular  arc.  Problems  of  Thrust -limited  rocloets  sub¬ 
jected  to  aerodynamic  forces  are  another  example  of  pursuit-evsision 
games  with  the  possibility  of  singular  or  intermediate  thrust  eurcs  for 
either  or  both  pursuer  and  evader.  In  the  next  section  we  formulate  a 
differential  game  problem  and  consider  cases  with  singular  arcs. 
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There  axe  two  players,  (P)  is  trying  to  minimize  J  hy  control 
n  and  (E)  is  trying  to  maximize  J  by  control  v.  It  is  assumed 
that  both  players  have  perfect  information  about  the  system,  and  also 
each  player  has  partial  control  over  the  game. 

The  problem  is  to  determine  u  and  v  such  that 

J(u*,v)  <J(u*,v*)  <  J(u,v*) 

holds.  The 'solutions  specified  by  u  .and  v  are  optimal  and  termed 
the  saddle  point  solution. 

u(*)  and  v(«)  respectively  belong  to  the  sets  U  euad  V,  The 
final  time  is  assumed  to  be  fixed,  h  and  L  are  scalar  fncfcloas 
aiid  ass-naed  to  be  smooth.  The  control  sets  are  defined  by 

U  *  {u(«)j'Uj^  is  piecewise  contlnuotas  in  tine  |u^(»)l  <  * 

(2. '+-5) 

V  *  (v(*);v.(«)  is  piecewise  contitiuous  in  time  |v. (•)!  <  » 

J  e  J 

Iq  ^  t  ^  t^  J  »  1,2, •..,r}  (2,4-6) 

the  initial  state  and  Inltiail  time  are  fixed  euid  final  state  is  free. 

In  order  to  find  the  saddle  point,  the  Hamiltonian  is  defined  as 
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where  X.  e  R  Is  Zagraaglan  miltipUer. 

Assuming  there  exists  a  seiddle  point  solution,  the  set  of 
necessary  conditions  for  the  saddle  point  solutions  are 


•* 


*  *  * 


(t)  >  f(x  ,u  ,v  ,t)  xCt^)  e  Xq  (2.14-9) 


•* 


Mt)  -  - 1 


-H 


(2. 1^-10) 


H(x  ,u  ,v,t)  ^  H(x  ,x  ,u  ,v  ,t)  i  H(x  ,u,v  ,t) 


ah(x(t  J  ,t  J 

x(0 - 


nr 


(2.1^-11) 


(2.14-12) 


The  saddle  point  solution  should  satisfy  (2.I4-II)  and  usually  is 
obtained  as 


u  »  arg  min  H(x  ,u,v  ,t) 


u 


V*  «  arg  max  H(x*,u*,v,\*,t) 


(2.1f.l3) 


(2.1f-ll^) 


In  the  case  that  Hamiltonian  is  linear  in  terms  of  ccisponents  of 
V  and  u  and  there  are  bounds  on  controls,  i.e. 


|u^|  ^  ^li  ^  *  1,2, ..,,m 

|Vjl  ^  ^  "  l»2,...,r 


(2.1V-15) 

(2.1V-16) 
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optimal  controls  can  te  expressed  as: 

Ui  »  -Kj^'jSgn  A  ,ti,v  ,t)  if  /  0 

1  >  ly2, •••^m 

Vj  “  if  HVj  0 

J  (2»^*— 17) 

assuming  all  ccmponents  of  u  and  v  are  independent  of  each  other. 

Note  that  similar  to  the  optimal  control  problems  there  may  exist 
conditions  such  that  Hu  and  Ht  or  both  be  equal  to  zero  for  some 
n<si>zero  time  interval  in  the  game.  Then  ve  will  face  the  problem  of 
singularity. 

Definition  2.5«  If  one  or  more  ccmponents  of  the  control  functions  u 
or  V  or  both  appear  linearly  in  the  Hamiltonian  defined  in  (2.V-8) 
and  there  exists  a  non-zero  subinterval  of  time  between  t^ 

and  t^  such  that  the  coefficient  of  at  least  one  of  the  control  com¬ 
ponents  is  zero  on  this  subinterval.  The  control  is  said  to  be  singular 
and  this  subinterval  is  said  to  be  a  singular  interval.  The  TAMrimm 
principle  (2.4-11)  provides  no  information  about  these  caatrols  and 
their  relationship  with  x  and  \  in  this  interval. 

All  the  definitions  in  Section  2.1  about  the  singularity  in  optimal 
control  problems  will  bold  for  singular  two-sided  problems  aiui 
Anderson^  has  derived  necessary  conditions  for  optimality  of  singular 
arcs  in  differential  games  which  are  exactly  the  saaM^  as  necessary 
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conditions  for  optimality  of  singular  control  problems.  Thus,  the 
analogrof  the  GLC  condition  in  the  two-sided  problems  eure 

i  0 

and  the  analysis  of  junctioi  points  carries  over  as  well. 

2.5  Derivation  of  Singular  Control.  For  simplicity  of  calcaation  a 
restricted  class  of  nonlinear  singular  differential  games  is  considered 
Find  the  saddle  point  solution  u  and  v  to  the  payoff 

J  =  h{x(tp)  (2.5-1) 

subject  to: 

x(t)  «  fj^Cx)^  +  f2(x,v)  (2.5-2) 

x(tQ)  -  Xq  given  (2.5-3) 

|u|  SK  (2.5-4) 

where  x  e  V  e  R*",  u  is  scalar,  f^  and  fg  are  n  x  1  vector 
value  functions  at  least  n  times  differentiable  with  respect  to  x, 
h  is  a  smooth  scalar  function,  and  t^  is  fixed.  Assiming  the  saddle 
point  solution  exists,  the  set  of  necessary  conditions  are  obtained  as 
the  foUoidng. 


(2.4-18) 

(2.4-19) 
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To  find  tbe  singular  control  in  this  internal  ve  ccxisider  (S.^-U) 
and  its  respective  tijne  derivatives 

^  (x,X.)  »  0  »>  «  0  (2.5-12) 

as  it  is  noticed  this  relationship  does  not  yield  any  information  about 
singular  control  u  .  So  we  taloe  the  time  derivative  of  (2.5-12)  sub- 

9 

stitutihg  from  (2.5-8)  and  (2.5-9)  ^or  x  and  \  and  ve  have 

dt  5u  ■  0  *>  \  (^3jc^2  "  ^2x^1^  “  ®  (2.5-13) 

(2.5-13)  still  does  not  yield  any  information  about  singular  control  u„. 

s 

Let 


+  \  (8^^2  "  ^2x®^  "  ^ 


T 

If  \  (^^2.  "  ^ix®^  ^  ^  singular  control 


\  “  ^2x®^ 

X  (^^1  "  ^Ix®^ 


(2.5-11*) 


(2.5-15) 


(2.5-16) 
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If 


°  (2.5-17) 

ve  continue  taking  time  derivatives  of  (2.5-17)  until  u  appears 
e^q^licitly  with  a  nonzero  coefficient.  In  most  porblems  of  interest 
u  appears  in  the  n^^  order  time  derivative  of  (2.5-12).  If  this 
control  is  optimal  then  it  satisfies  the  strengthened  GIC  condition. 


2.6  Linear  Quadratic  Singular  Differential  Game  with  Bound  on  Control. 
A  class  of  generalized  linear  pursuit -evasion  differential  game  is 
described  by  the  following  state  equations  and  performance  index 


(2.6-1) 


subject  to: 


x(t)  «  Ax  +  Bu  +  Cv 

(2.6-2) 

3c(to)  -  *0 

(2.6-3) 

|u|  siK 

(2.6-1^) 

where 

X  n  X  1  state  vector 

u  scalar 
V  r  X  r  vector 
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A»  B,  C  are  coatlnuous  time  varying  matrices  vlth  compatitle  sizes. 


S  n  X  n  continuaus  time  varying  positive  semideflnlte  matrix. 

Q  n  X  n  cantinuous  time  varying  positive  definite  matrix. 

R  ■  r  X  r  continuous  time  varying  positive  definite  'bounded  symmetric 

matrix. 

t^  fixed  final  time. 

In  this  class  of  problems  there  are  two  players.  Pursuer ,  (F) 
seeks  control  u  to  minimize  J,  emd  evader  (E)  tries  to  find  con¬ 
trol  over  his  oim  state. 

In  the  next  section  we  will  show  that  this  problem  under  seme 
conditions  possesses  a  unique  saddle  point  solution. 

2.6.1.  Saddle  Point  Conditions.  We  now  impose  some  conditions  on  the 
siatrlces  A,  C,  Q  and  R  to  assure  the  existence  of  the  saddle 
point. 

In  this  class  of  linear  quadratic  differential  game  for  the 
existence  of  a  unique  saddle  point,  the  performance  index  should  be 
strictly  convex  with  respect  to  u  for  any  fixed  v  and  strictly 
concave  with  respect  to  v  for  any  fixed  m. 

Strict  convexity  of  J  with  respect  to  u  is  easily  verified 
"by  the  given  assumption  of  positive  definiteness  of  Q  and  controll- 
ahlUty  of  the  system  (AB)  and  since  u  belongs  to  a  set  of  convex, 
closed  and  bounded  there  will  be  a  unique  lll^T^^nBHll  fo]»  >wy  given  v. 

To  establish  concavity  of  J  with  respect  to  v  some  conditions 
are  required  so  that  concavity  of  the  second  term  dominates  convexity 
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(2.6-10) 


IKt)||2  dt) 


1/2 


Satisfaction  of  (2.6-7)  is  equivalent  to  shearing  that 
t^  t  t 

J  (  J*  'P(t,T)c(T)R“^^^  w(T)dT)^Q(  J  Cp(t,T)c(T)R"^^^  w(T)dT)dt 


■to  0 


r^f 


-  j  V'‘‘(T)w(T)dT  <  0  (2.6-11) 


We  normaliae  w(t)  by  defining 


5(t) 


W(T 


(2.6-12) 


so 


mil  - 1 


(2.6-13) 


Assuming  |(w||  /  0  and  dividing  (2,6-11)  by  |jwjL  and  using  the 


relationship  (2.6-12)  it  is  enou^  to  show 

t 


‘f  * 


J  (  J  'P(t,T)c(T)R"^'^^  §(T)dT)Q(  J  <P(t,T)c(T)R"^^^  S(T)dT)dt  <1 

^0 

(2.6-llf) 


^0  ^0 


Using  norm  inequalities 


t-  t 

i  f* 


If  (  J  ^(t,t)c(T)R"^'^  5(T)dT)\(  J  cp(t,T)c(T)R'^''^  5( 

I  •'t„  •^0  *0 

==  J  .  (  !|9(t,T)c(T)R“^/^ll^  \\li^)\\j^  llQlIj^  t  dt)dt 


5(T)dT)dt 


^0-  ^0 


J  (  j'  ||9(t,T)c(T)R‘^/^|l^dT  J  ||S(T)||^dTl|Q|yt  dt 


^0  ^0 


J  (  J  ||<P(t,T)c(T)R"^^llJdT||(i||3_)t  dt 


(2.6-15) 


Therefore 


*0  *0 


t  t 


I  J  (  J  'P(‘t»T)c(T)R“^^5(T)dT)^Q(  J  <P(t,T)c(T)R"^'^^§(T)dT)dt 
llsihl  I  •'t,  •'t.  *'t^ 


-0  “0 


t 

J  (t  -  V  J  ||tp(t,T)c:T)R‘^/^||^dT||Q||3^dt 


vhere 


M.  •  „ 


(2.6-16) 


(2.6-17) 


So  if 


f 

J  (*-V  J  IN>(t,T)c(T)R’^/^||^dT||(i||3_dt  <  1  (2.6-18) 


Then 


Ul 


-t. 

(  J  <P(t,T)c(T)v(T)dT)’^Q( 

^  *0  t 

-  J  v'^(t)Rw(t)dt  <  0  (2.6-19) 

^0 

Note  condition  (2.6-18)  iiapUes  the  negative  definiteness  of  J 
with  respect  to  v  which  easily  results  in  the  concavity  of  J  with 
respect  to  v. 

Besides  (2.6-18)  we  must  show  that .  J  Is  radially  unbounded  in  v 
in  order  to  establish  the  existence  of  saddle  points. 

It  can  be  verified  that  J(u,v)  -o#  as  |(v||  -‘«o  for  any  fixed  u 
by  the  following  relationships 
t^  t  t 

J  (  J  9(t,T)c(T)v(T)dT)\(  J  <P(t,T)c(T)v(T)dT)dt 

^0  ^0  *0 

tf  t 

^  J  -  V  I  II^(t,T)c(T)E“^^¥iT||(i||^dt!lw|l| 

■*^0  ■ .  ■*^0  ^ 

(2.6-20) 

and 

*f  0 

J  v(t)Rv{t)dt  -  llwlll  (2.6-21) 

*0 

Subtrcu:tlng  (2.6-20)  from  (2.6-I9)  ^  will  obtain 
tf  ^t 

J‘(v)  ^(1-  J  J  l|!p(t,T)G(T)E'^^^||ldTl|Ci||^dt)||wf  (2.6-22) 

*0  *0 


t 

J  <P(t,T)c(T)v(T)dT)dt 

t^ 

0 


k2 


Frcm  (2.6-18),  the  abo\re  parenthesis  is  a  positive  constant  number 
for  any  fixed  t^.  Since  R  is  bounded  ||v||  -• «»  Implies  that 
llirllg  -*  0#  and  from  J*(v)  ^  K  >  0  J’Cv)  "*  *•  So  it  is  con¬ 

cluded  that  J(u,v)  is  radially  unbounded  for  any  fixed  u. 

Since  for  this  class  of  problems  all  the  required  conditions  for 
the  existence  of  a  saddle  point  hold,  so  (2.6-18)  is  a  sufficient  con¬ 
dition  for  the  existence  of  a  saddle  point. 

In  a  more  general  case  when  S  /  0  the  term  outside  the  integral 
of  the  performance  index  will  yield  a  similar  term 

tf 

-  ■^0^  J  (2.6-23) 

ehlch  is  added  to  the  left  side  of  (2,6-18). 


1^3 


.  „/****»  .  *  *  *. 
H(x  ,\  ,u  ,v)  s  H(x  ,u  ,v  )  i  H(x  ,\  ,u,v  ) 


(2.6-28) 


X.  (tp  «  sx  (tp 


(2.6-29) 


The  strengthened  QJC  condition  for  the  optimality  of  the  singular 


control  of  the  first  order 


(2.6-30) 


is  satisfied. 


From  (2.6-27)  control  v  is  obtedned  as 


-E-^V 


(2.6-31) 


Substituting  for  r  from  (2.6-31)  in  the  state  equaticns,  the  set 
of  tvo  point  boundary  value  problem  viU  be 


x(t)  «  Ax  +  Bu  -  CR"^c\ 


\(t)  «  -Qx  -  a\ 


V  "  *0 


\(tp  «  Sx(tp 


The  control  u  is  obtained  as 


(2.6-32) 


(2.6-33) 


(2.6-34) 


(2.6-35) 
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+K 


b\  <  0  f or  t  e 

b\  >0  for  t^e  [t^jtg] 

b\  m  0  for  <  t  <  tg 

(2.6-36) 

As  lias  been  mentioned  in  the  interval  of  time  that  dlf(x,K,u)  /du 
T 

e  B  X.  «  0,  the  control  is  singular  and  obtained  by  taking  successive 

time  derivatives  of  dH/du  until  u  explicitly  appears.  To  calcu- 

s 

late  singular  control  u_(x,\)  ve  find 

s 

^.b\.o  (2.6.jr) 

~  ^  -  b\  +  b\*  «  -bV  +  (b'^-B^A)\  -  0  for  t^  <  t  <  tg 

(2.6-38) 

T 

Continue  tsdElng  time  derivatives  of  (2.6-38)  since  B  QB  >  0  then  the 
singular  control  appears  in  the  next  relationship  so  that 

Ug  •  Mx  +  N\  '•'1  <  ^  "^2  (2.6-39) 

vhere 

M  -  (B^QB)"^(B^AQ  -  i\  +  B^Q  -  i\  -  b’^QA)  (2.6-U) 

N  «  +  B^b\  -  b'^A^  -  +  b’aA^)  (2.6-42) 
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if 

U  s  /  -K  if 

undetermined  if 


By  substitution  of  uon-singular  and  singular  controls  in  the  state 
equations  ve  will  obtain 

(  x(t)  -  Ax  -  CR'^C*\  +  BK 
(  \(t)  ■  -Qx  -"A\ 

which  hold  on  non«4lngular  interval  and 

x(t)  -  (A  +  a4)x  +  (BR  -  CR"^C^)\  (2.6-V5) 

I  \(t)  -  -  Q?c  -  a\  (2.6.U6) 

trtilch  hold  on  singular  Intervals.  Ih  Chapter  3  ^  vlU  show  how  to 
treat  these  sets  of  equations  with  given  and  obtained  boundary  condi- 
tlcns  as  a  znultlpolnt  boundary  value  problem. 

Remark.  According  to  Reference  (I4-)  the  existence  of  an  optimal  v  can 
be  verified  throu^  an  auxiliary  problem  assuming  u  (t)  is  an  optimal 
open  loop  solution 

*f 

max  {|  x'^(tpsx(tp  +  J  (x'^Qx  -  v^Rv)dt}  (2.7-4?) 

subject  to: 


(2.6-43) 

(2.6-44) 


x(t)  ■  Ax  +  Bu  +  Cv 


(2.6-46) 
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u'b  u  (t)  a  time  function 


(2.6-49) 


The  eolation  for  v  Is 

▼(t)  s  R"V(KSc  +  s)  (2.6-50) 

where 

a(t)  +  (KGR"^*^  ■••a’^)s  +  KBu*  8(tp  »  0  (2.6-51) 

K(t)  +  +  KA  +  KCR’^^K  +  Q  »  0  (K(t^)  -  S 

(2.6-52) 

Since  u  (t)  is  hounded  from  the  ahov«  we  conclude  that  there 
* 

win  he  an  optimal  v  if  there  exists  a  finite  solution  to  the  Rlcatti 

equation  (2.6-52)  for  0  <  t  <  t^.  In  such  a  way  the  existence  of  a 

ssuidle  point  to  the  game  (2.1-1)  -  (2.1-4)  ceui  he  verified  since  there 
* 

exists  a  unique  u  in  this  problem. 


CH/LFEEB  3 


XHE  CCMHJT/lTICir  QF.SIRGUIAB  OFFlMAIi  CCRTBOL  AND  DIFFEBEXTEIAL  GAMES 

Despite  the  asiount  of  interest  In  singular  control  problems 
developnent  of  the  computational  aispect  of  this  problem  requires  more 
attention.  Due  to  the  control  .boinds  and  control  discontinuities  some 
computational  and  analytical  difficulties  are  encountered.  To  deal 
vlth  these  kinds  of  difficulties  some  special  considerations  should  be 
taken  In  singular  problems. 

Fagurek  and  Woodslde^^^  (1968)  have  presented  a  direct  method 
and  applied  a  con;|ugate  gradient  method  in  function  space  for  optimal 
ccmtrol  problems  with  bounds  on  controls  and  have  solved  a  problem 
with  singular  ares.  Ih  this  technique  the  region  of  saturation  should 
be  guessed  a  priori  and  acme  procedure  is  devised  to  improve  this 
guess  at  ea<±  iteration.  Althou^  the  rate  of  convergence  for  some 
problems  has  been  almost  good,  the  solution  obtained  by  this  tedmlque 
Is  not  accurate.  Also  Junctloi  points  between  saturated  and  unsaturated 
controls  are  not  obtained  In  their  exact  locations.  Eo  and  Stevens' 
(1971)  applied  gradient  methods  to  determine  the  optimal  heat  transfer 
coefficient  distribution  along  a  tubular  reactor.  This  method  handles 
both  singular  and  bang  bang  arcs,  but  It  obtains  an  approximate  solu¬ 
tion  and  In  the  presence  of  singular-  arcs  has  a  very  slow  rate  of  con- 
-vergence  when  It  gets  close  to  the  optimum.  Jacobson  et  al.  have 
transfonned  the  singular  optimal  control  problem  to  a  noi-singular 
problem  by  adding  a  quadratic  Integral  function  of  ccx^-trol  to  the  per¬ 
formance  Index.  This  integral  function  Is  multiplied  by  a  coefficient 
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which  tends  to  zero  iteratively  during  the  computatioa  of  non-singular 

control,  so  that,  the  solution  converges  to  the  original  singular  one. 

•  f  i9) 

Jacobson,  Gershwin  and'Lele  (1970)  have  applied  differential  dynam¬ 
ic  programming  (Jacobsen  1968^^^^)  to  the  rekilting  non -singular  prob¬ 
lem  and  have  solved  severtkl  examples.  This  technique  has  given  seme 
satisfactory  results,  but  often  cos^tatlonal  difficulties  arise  when 
the  coefficient  of  Integral  tends  to  zero.  In  the  same  reference  a 
variant  of  the  method  called  epailon-aplha  (e-o)  algorithm,  has  been 
proposed  by  the  authors  to  overcome  this  difficulty.  With  this  altern¬ 
ative  method  another  parameter  or  la  considered  and  the  coefficient 
of  the  integral  does  not  have  to  approach  to  zero.  However,  it  still 

does  not  guarantee  the  convergence  of  the  singular  optimal  cwtrol  and 

f  35) 

accurate  resvilts  cannot  be  obtained.  Ih  1972  Edgar  and  lapidus^*^^^ 
used  the  method  of  Jacobson  et  al^  together  with  differential  dynamic 
programaing  and  penalty  function,  and  applied  that  to  discretized 
versions  of  the  problem.  First  they  developed  the  sQgorlthm  for  linear 
control  problems,  then  they  extended  the  technique  to  non-linear  prob- 
lezks.  This  technique  has  the  same  computational  difficulties  as  that  of 
the  method  of  Jacobson  et  al.  The  degree  of  accuracy  is  low,  especially 
since  the  problem  is  descrltized,  the  exact  switching  time  cannot  be 
obtained.  Also  this  method  is  time  consuming  for  non-linear  and  seme 
linear  problems.  This  technique  has  the  advantage  of  capability  of 
handling  problems  with  hlgii  dimensions.  Anderson'^  (1972)  proposed 
an  indirect  method  for  computations  of  singular  optimal  control  probleos 
with  first  orders  of  singularity.  This  technique  Iterates  on  initial 
costates  and  a  prescribed  Terminal  Error  Function  is  minimized  with 


respect  to  initial  costates  by  a  search  technique.  The  sequence  of 
non-singular  and  singular  controls  are  esthnated  a  priori  and  at  each 
iteration  switching  conditions  are  checked  and  adjusted.  The  accuracy 
of  this  technique  is  good  and  the  rate  of  convergence  of  the  method 

depends  on  the  rate  of  convergence  of  the  search  technique. 

yg^(37)  (1975)  applied  quasilinearization  together  with  (e-Qf) 

algorithms  of  Jacobson  to  solve  singular  problems.  This  method  can 
have  a  quick  rate  of  convergence  if  the  coefficient  of  the  added 
quadratic  Integral  is  too  small.  -  But,  as  this  coefficient  gets  smaller 
the  likelihood  of  convergence  gets  smaller  too.  This  method  has  a 
simple  programming  and  faster  computational  speed  than  the  same  method 
using  differential  dynamic  programming.  In  1^6  Edge  and  Powers^ 
in  a  function  space  quasl-Rewton  algorithm  applied  Davidon  and  Broyden 
algorithms  to  optimal  control  problems  with  bounded  and  singular  eircs. 
Examples  indicated  that  this  method  is  more  accurate  than  gradient  and 
conjugate  gradients,  but  still  the  obtained  solutions  in  an  approxima- 

( 29) 

tlon  of  the  real  optimal  solutions.  In  1973  Aly^  and  Chan  applied  a 
modified  quasillnearlzatlon  technique  to  totally  singular  problems  and 
in  1978  Aly^^^^  applied  the  same  technique  to  partially  singular  prob¬ 
lems  with  first  order  singularity.  In  this  technique  the  sequence  of 
controls  are  estimated  a  priori  and  at  each  iteration  the  initial  co¬ 
states  are  adjusted  to  satisfy  the  j\mction  conditions.  This  adjustment 
may  accelerate  or  may  decelerate  the  rate  of  convergence  of  the  mod¬ 
ified  quaslllnearizatlon  used  in  this  method. 

In  the  following  section  we  have  presented  an  indirect  method  with 
two  approaches.  The  first  approach  solves  certain  groups  of  problems 
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irtiich  can  provide  n  conditions  at  the  junction  of  singular  and  non- 
singular  controls.  In  these  cases  we  solve  a  set  of  mltipoint  houndary 
value  problems  by  iterating  only  on  switching  time.  This  approach  has 
been  proposed  for  linear  problems,  thus  .it  is  extended  to  ncmlineax 
problems.  The  second  approach  can  solve  linear  singular  problems  with 
singularities  of  order  of  one  to  n.  In  this  approach  an  iterative  pro¬ 
cedure  which  iterates  on  switching  time  and  Initial  costates  is  used. 
This  method  can  directly  solve  singular  differential  game  problems. 

3.1  Numerical  Techniques 

In  totally  singular  problems,  since  there  is  no  discontinuity  of 
the  control  and  junction  points^  solutions  to  the  set  of  TPBVP  obtained 
frail  the  necessary  conditions  for  the  saddle  point  may  be  obtained  by 
some  of  the  numerical  techniques  In  the  liteiuture.  In  partially  sing¬ 
ular  problems  special  considerations  should  be  teJsen  for  the  junction 
conditions  and  discontinuity  of  the  control.  So  far,  there  has  not 
been  found  a  general  sufficient  condition  for  optimality  cf  the  petr- 
tially  singular  control.  If  there  exists  a  single  solution  to  the 
problnn  satisfaction  of  the  necessary  conditions  is  enough  for  the 
optimality.  But  if  there  are  a  finite  number  of  solutions,  some  com¬ 
parison  should  be  made  In  order  to  find  the  optimal  one^^^^.  In  the 
proposed  technique  we  estimate  the  sequence  of  singular  and  non- 
singular  controls  in  time  intervals  of  the  game.  In  the  simplest  case 
we  consider  only  one  junction  point.  For  this  case  a  finite  nuaiber  of 
possible  sequences  may  occur.  We  consider  two  major  different  cases. 


Case  1.  Game  starts  with  a  aon-slngular  arc  and  at  soaie  tine 
switches  to  a  singular  arc  and  terminates  on  the  singular  arc  at  time 
t^. 

Case  2.  Game  starts  with  a  control  on  one  bound  and  then  at  some  time 
t^  the  control  switches  to  the  other  bound  i^lch  is  the  bang  bang 
case.  For  dealing  with  Junction  points.  It  is  enough  to  consider 
these  two  cases  and  other  cases  are  basically  the  same,  but  more 
tedious  when  the  number  of  switchings  are  Increased. 

In  Case  1  since  control  u  is  bounded,  discontinuity  of  u  will 
not  cause  dlsccxitlnulty  In  the  states  and  costates,  however,  it  may 
create  some  comer  points  at  the  Junction  of  singular  and  non-singular 
arcs.  So,  the  following  relationships  hold  at  each  Junction  point 

»(tp  -  x(t*)  (3.1-1) 

\(tp  -  \(t^)  (3.1-2) 

where  ts  and  ts  respectively  represent  time  Just  before  and  t  ime 
Just  after  the  switching  time  ts.  Figures  (3.1-a)  -  (3.1-d)  a 
possible  scheme  of  controls,  states  and  costates  for  a  differential 
game. 

The  following' relationship  holds  at  any  point  along 
^H/^(x(t),\(t),t)  -  0 

^  ^  (x(t),\(t),t)  -  0  t^  s  t  s  t^  (3.1-3) 
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the  singular  arc  Including  t***.  Continuity  condition  (3.1«l)  and 
(3.1-2)  together  with  (3*1-3)  inplies  that  (3.1-3)  hold  at  t‘  on  the 

B 

non-singular  arc  and  provides  n  conditions  at  the  Junction  point.  So 
a  set  of  multipoint  boundary  value  problems  is  fomed.  Also  ve  define 
a  Terminal  Error  Vector  Function  TEF  as 

TEF  .  \(tp  -  sx(tp  (3.1-^) 

vhlch  is  a  boundary  condition.  Since  the  time  t^  is  unhnovm  ve 

establish  a  procedure  that  iterates  on  t  and  drives  euclidean  norm 

8 

n(t^)  •  |flEF||  to  zero,  so  that  all  the  necessary  and  boundary  condi- 

tlons  for  optimality  are  satisfied.  In  oMer  to  derive  the  iterative 

relationship  for  the  switching  time  t^  we  purturb  switching  time 

s 

around  a  nrmlnal  value  tf .  So,  the  final  states  and  costates  are  pur- 

s 

turbed  as  much  as  i^(t^)  and  ^\(tp.  Figure  (3.2-a,b)  shows  the 
change  of  final  state  and  eostate  due  to  the  change  of  switching  time. 
If 

TEP^  «  \^(tp  -  Sx^(tp  (3*1-5) 

where  x^(t^)  and  x^(tp  are  the  final  state  and  costate  obtained  by 

the  choice  of  t^..  So 
s 

TEF  -  TEP^  +  ATEP  •  \^(tp  +  ^\(tp  -  Sx^(tj.)  -  S13c  (t^ 

(3.1-6) 

We  express .  ^(t^  and  Ax.(t^)  as  a  fhnctlon  of  At^  where 

4t,  -  t,  -  (3.1-7) 
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for  this  porpose  we  solve  (2.6«44)  and  (2.6-4^)  for  the  interval 
t  c  [tQ,t^] 


p(t.)- 

■  t»(t  1+  tiKtJl 

•  °  Lx(to)J  “ 

(3*1-8) 

r  A  -oa-Vl 

(3*1-9) 

^(t^.tQ)  -  I 

(3*1-10) 

and 

TKt.)  . 

“  J  1 

to  L  0  J 

(3*1-11) 

solving  (2.6-U6)  and  (2,6-U7)  we  get 

rx(tp-j 

Lx(vJ 

r  *^t.)  1 

-  [^r(t  ,t )] 

^  ®  L  \(tg)  J 

(3*1-12) 

i(ber« 

- 

TA-tai  ai-cR'^c’^'i 

■  L  ^  J 

(3*1-13) 

t(t^,tp  -  I 

(3*1-11^) 

According  to  (3*1-7)  the  following  relationships  hold 
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X(tg)  -  X(tl)  +  Mtg) 


>w(tg)  -  \(ti)  +  AX(tg) 

Tl(tg)  -  Tl(ti)  +  ^Tl(tg) 

♦(t-jt  )  «  ♦(t«,ti)  +AHr(t  ) 


(3.1.15) 


Trtiere  ^Tl(t  ),  A$(t  )  and  At(t_)  can  be  expressed  as  a  function  of 

B  a  o 


Frcoi  (3,1-8)  and  (3,1-12)  and  (3*1-15)  the  following  relationship 
can  be  obtained. 


l,tQ)]  *  +AI(tg)r  0  +AT1(t 

l-Ax.(tp,)J  ^X.(0-l 


(3.1-17) 


Expressing  Ai(t^),  Ai|f(tJ),  AT|(t^)  and  ^\('fcQ)  as  functions  of 

Atg  to  the  first  order,  we  can  derive  Ax(t^)  and  A\(tp  in  tenas 

of  Atg.  By  substitution  ffoti  (3,l-l8)  and  (3.I-I9)  in  (3.1-6)  we  can 

obtain  TEP  an^  St™  ®a^  tg.  Therefore  we  ceua  obtain  several 

s 

Iterative  relationships  to  derive  TEF  to  zero. 


The  switching  time  can  be  obtained  as 


s 


(3.1-17) 


Or  in  the  least  square  sense 


.i+1 


•“ST” 


TEP. 


(3.1-18) 


^rtiere 


9TEPt 


(3.1-19) 


The  details  of  the  derlvatlcm  are  given  in  Appnedlx  B.  To  see  how 
this  technlqvie  la  accomplished,  we  outline  the  algorithm  as  it  would 
be  executed  if  a  digital  computer  were  used. 


Algorithm 

Step  1.  Select  a  nominal  8vit<*ing  time  t^,  ^0  <  t^  <  t^  assuming 

* 

this  choice  is  sufficiently  close  to  the  optimal  switching  time  t  . 

s 

let  the  iteration  index  1  be  zero. 

Step  2.  Integrate  the  state  and  costate  equations  (2. 6-^4-)  and  (2. 6-^3) 

frcm  tn  to  t/  as  a  set  of  TPB7P  with  boundary  conditions  (2.6-3) 

0  S 

and  (3. 1-3)  and  store  x(tM  and  \(t^). 

S  9 

step  3.  Ihtegrate  the  state  of  costate  equations  (2.6-U6)  and  (2.6>V7’) 
from  t^  to  t.  a5£set  of  initial  value  problems  with  initial  condition 

.8  IT 
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|!TEF(t  J)  II  s  -Y 


(3.1-20) 


vhere  7  Is  preselected  positive  constant  the  problem  is  solved, 

otherwise  compute  t^"^^  from  (3,1-17)  and  repeat  the  procedure  from 

s 

step  2  to  Step  W. 

3.2  Extension  of  the  Htunerlcal  Algorithm  to  a  Class  of  Non-Idnear  Games 
Let  the  set  of  state  and  costate  e<juations  obtained  from  the  appli¬ 
cation  of  the  necessary  condition  for  the  saddle  point  be 

r  x(t)  •  f(x,\,t) 

1  \(t)  -  g(x,\,t) 

r  •*  1 

for  non-singular  Interval  [tf.,t^J  and  for  singular  interval 

u  s 

be 

{±  -  f(x,X,t)  (3.2-3) 

\  m  g(x,\,t)  (3.2-J4-) 

x(tQ)  .  Xq  given  (3.2-5) 


(3.2-1) 

(3.2-2) 


ah(x(0) 
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If  the  switching  time  Is  perturbed  as  such  as  At  ■  t  -ti,  then, 

M  8  8 


TEF  m  TEF^  +  ATEF 


(3.2-17) 


where  AZEF  Is  the  Increment  of  TEF  due  to  the  increment  At  .  We 

8 

can  express  the  foUowlng  relationship 


TEP  .  \*(0  +  6\(0  -  ^  ^  6X(0  +  H.O.T. 


(3.2-lB) 


By  solving  (3*2-12)  -  (3*2-14)  and  substituting  in  (3.2-18)  for 

8x(tp  and  fi\(tp  in  terms  of  At^  we  find  ATE?  as  a  function  of 

At„  so  we  will  be  able  to  find  . 

The  Iterative  formula  tar  the  switching  time  is  the  same  as 

(3.1-17)  or  (3.1-18)  and  the  derivation  is  given  in  Appendix  B. 

The  procedure  is  carried  out  on  a  digital  computer  similar  to  the 

linear  case,  the  only  difference  is  the  computation  of  which  is 

done  throu^  linearization  according  to  the  Appendix  B. 

In  the  case  that  there  are  two  switching  points  the  sequence  of 

singular  and  non-singular  controls  are  estimated.  For  example,  assume 

the  game  starts  witii  a  non-singular  arc  and  at  time  t  switches  to 

®1 

a  singular  arc  and  at  time  t  switches  to  a  non-singular  arc  and  the 

®2 

game  terminates  on  the  non-singular  arc.  t_  and  t  are  inttiaiQir guessed 

®1-  ®2 

and  it  is  tried  to  drive  TE?  to  zero  with  respect  to  t.  and  t, 

®1  ®2 

which  are  Independent  of  each  other. 


3.3  Second  Approach 

In  some  hl^  dimensional  problems  the  number  of  Jvinctlon  conditions 
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may  be  m  <  n.  In  such  a  case  the  first  approeich  shouli  be  modified 

Terminal  Error  Function  should  be  driven  to  zero  by  iterating  on 

the  switching  times  and  n-m  initial  costates,  i.e., 

j  s  1,2, ,,.,n-m.  For  the  class  of  problems  with  linear  state  equations 

TEF  is  expressed  as  a  function  of  initial  costates  \{^q)  BOd 

switching  time.  In  this  approach  for  a  computed  switching  time  t|, 

initial  costates  are  also  computed  to  minimize  ^£(x.(bQ),tp  * 

llTEpjl^  at  the  1^^  iteration,  and  then  at  each  t^  the  gradient  of 

Min^^^  j  llTEFll  with  respect  to  t^  is  obtained.  In  order  to  satisfy 

the  ;Junctiai  conditioia,  at  each  step  of  minimization  m  constraints 

on  initial  costates  is  considered,  fiy  finding  the  minlTPan  value  of  h 

aiid  its  gradient  with  respect  to  t  at  tach  t^  we  will  he  able  to 

find  a  new  t^  irtiich  yields  a  ainaller  A,  i.e.,  (by  Newton  method), 
s 

This  procedure  is  continued  until  HTEFH  «0,  in  whi(ih  case  the 
problem  is  solved. 

The  value  of  the  error  at  final  time  for  each  iteration  as  a 
function  of  switching  time  and  initial  costates  is  obtained  as 


«  P(t^)  +  Q(ti)\^tQ)  +  >.^^tQ)^(t^)\i{tQ)  (3.3-1) 


Where  P(t^),  5i(t^)  and  R(t^)  are  functions  of  switching  time  given 

S  3  S 

in  Appendix  C.  Also  m  conditions  at  the  Junction  point  yields  m 
constraint  relatitfiships  on  the  components  of  as 


Is  an  m 


where  W,(tJ)  and  Wo(tf)  are  m  X  n  matrices  and  T(<*) 

X  s  2  S  9 

vector  specified  In  Appendix  C. 

The  following  Iterative  relationship  may  be  used  to  find  the 
optimal  switching  time 

-1 

t^"^^  -  tf-  +  n  3  [Min  n]  I  (3.3-3) 

The  derivation  of  h  is-  shown  in  Appendix  C. 

s 

The  following  algorithm  is  the  outline  of  the  steps  required  to 
carry  out  this  approach. 


Algorithm 

Step  1.  Select  a  nominal  switching  time  ^  ^s  ^f 
close  to  the  optimal  switching  time  set  i  ■  0. 

Step  2.  Compute  )  M\(tQ),t^)  ftom  (3.3-1)  and  (3.3-2). 

Matrices  R(tb,  ^(t^),  P(tb,  W(t^),  W«(tb  and  T(tJ)  should  be 

S  S  S  S  C  9  9 

computed  a  priori. 


Step  3.  If 

nln  *7 

x(to)  ° 

where  y  is  a  preselected  positive  scalar  number,  the  problem  is 
solved,  otherwise  go  to  Step 


Step  4.  Compute  [mln^^^  j  ^s^^ 

8  0  S 

(3.3-3)  so  to  Step  2,  repeat  the  procedure. 


from 
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3.U  PftattireB  of  the  Numerical  Algorithms 

The  Important  features  of  the  algorithms  are  ais  follows: 

1.  Initial  Guess.  To  begin  the  procedure  a  guess  for  initial 
switching  time  shauM  be  made.  This  guess  is  usually  based  on  the 
^lysical  nature  of  the  problem. 

2.  Storage  Requirement.  For  lineEur  problems  only  initial  state 
and  terminal  conditions  and  the  values  of  states  and  costates  at  Junc¬ 
tion  points  should  be  stored.  For  non-llneetr  cases  each  tnijectory 
obtained  at  each  iteration  is  stored  as  a  linearizing  trajectory  for 
the  next  iteration. 


3*  Convergence.  Since  the  method  has  the  characteristic  of  the 

Newton  method,  if  tf  is  sufficiently  close  to  the  optimal  t„  the 

s  s 

method  will  generally  converge  quite  rapidly.  However,  if  the  initial 
guess  is  very  poor,  the  method  may  not  converge  at  all. 

CcmputatiCTis  Required.  In  the  first  approach  at  each  itera¬ 
tion  ve  need  to  invert  an  n  X  n  matrix  and  solve  a  set  of  TFERTF  and 
in  the  second  approach  we  have  to  solve  a  non-lineeu*  programming  prob¬ 
lem  at  each  iteration. 


3.  Stopping  Criterion.  In  problems  with  fixed  final  states  the 
procedure  is  exactly  the  same  ais  free  final  state.  In  this  case  the 
tezminal  error  function  will  be  defined  as 


TEP  •  x(tp 


X 


f 


where  x^  is  a  given  vector  for  final  states  and  the  procedure 
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terminates  if  ||TEF||  ^  Y' 


Remark.  The  second  approach  can  directly  handle  problems  in  vhlch 
the  sequence  of  control  Is  bang  bang. 

3.5  Rtanerlcal  Kxample. 

Ccmslder  the  class  of  generalized  prusult -evasion  problems 
(2.6-1)  -  (2.6-4)  where  S  -  0,  Q  »  [J  J],  R  -  100,  A  -  [q  J],  B  -  [J], 

C  -  [J],  t©  -  0,  t^  .  3.17,  x(0)  -  [o;f^06^  and  K  -  1.  Assuming 
both  players  have  perfect  infoimatlan  and  measurements  of  the  output  and 
parameters  of  the  system  and 

y(t)  -  x(t)  (3.5-1) 

for  this  problem.  Condition 

J  t(  r  ||eA(*-'f)  CR'^^ll£dT)|ft||;Ldt  -  .21340  <  1 
•^0  *^0 

(3.5-2) 

and  all  other  required  conditions  for  the  existence  and  uniqueness  of 
the  saddle  point  are  satisfied.  For  given  matrices  and  t^  >  3.17  any 
R  2  21.3  satisfies  (3.3-2). 

Table  (3.I)  shows  the  solutlan  to  the  matrix  Rlcatti  equation 
(  2.6-32  )  which  is  finite  in  0  ^  t  ^  3. 17  is  an  alternative  to 
(3.5-2)  for  the  uniqueness  and  the  existence  of  the  saddle  point.  For 
the  necessary  conditions.'  Define  Hamiltonian 

12  12  IfV*)  o 

H-^x^+jXg-  V  +  \jXg  +  x.gU  +  \gV  (3.5-3) 
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V“  *'^v 


0*0 

e*i<soo 

0*25000 
0*37500 
0*500CQ 
0*02500 
0*  75000 
1*00000 
1*25000 
1*50000 
1*75000 
2*  00000 
1*25000 
2*500S0 
2*75000 
3* 00 oca 
3*25000 
3*00000 
3*75000 
^•07500 
0*00030 
4*12500 
4*25000 
4*37500 
4*50000 
4*02500 
4*40750 
4*75000 
4*01250 
4*07500 
4*53750 
5*00000 
5*00250 
5*12500 
5*15025 
5*10750 
5*  2I07S 
5* 25000 
5.20125 
5*31250 
5*34375 
5*37500 
5. 39003 
5*40025 
5*42100 
5*43750 
5.45313 
.  5*40075 
5*45430 
5*50000 


0*0 

0*12500 
0*25000 
0*37500 
0*50002 
0*02505 
0*75012 
1*00050 
1*25153 
1*50304 
1*75030 
2*01044 
2*27990 
2.55159 
2*03403 
3* 1 3400 
3*45702 
3*01443 
4*21920 
.4*44059 
4*09505 
4*97102 
5*20220 
5*03071 
O*  04000 
0*53059 
0*02130 
7* 13570 
7.40020 
7*00741 
5*34407 
0*07297 
9*49432 
10*23054 
10*00522 
11*14100 
11*07201 
12*27000 
12*94092 
13*72521 
14*02299 
15*07390 
1 0*  20970 
10*92152 
17*03751 
10*42700 
19*30490 
20*20309 
21 *30398 
22*02090 


0*0 

0*00701 
0*03125 
0*07034 
0*12500 
0*19552 
0*20171 
0*50102 
0*70571 
1*13550 
1*55370 
2*04435 
2*01340 
3*20991 
4*02727 
4*90403 
5*93045 
7*14721 
0*02019 
9*40300 
10*45399 
11*9501 1 
12*02333 
14*29990 
10*04901 
10*10179 
19*39903 
20*77002 
22*33407 
24*10900 
20*15179 
20*52093 
31*33450 
34*70000 
30*04920 
30*01002 
41*24223 
O*  9753 1 
47*07959 
SC *03733 
54*79710 
59*90913 
02*32409 
09*32309 
00*01041 
72*2  9750 
70*29770 
00*00933 
05*00049 
91  *02273 


0.0 

0*00701 
0*03125 
0*07034 
0*12500 
0*19552 
0*28171 
0*90102 
0.70971 
1*13950 
1.95370 
2*04435 
2.01340 
3.20997 
4.02 727 
4.90403 
5*93045 
7.14721 
0.02019 
9*40300 
10*45359 
11 *5501 1 
12*02333 
14*29990 
10.04901 
10*10179 
19*39503 
20*77002 
22*33407 
24,10900 
20*15175 
20*52593 
31*33490 
34*70000 
30*04920 
30*01002 
41*24223 
43*97931 
47.07959 
50*03733 
54*79710 
59*50513 
02*32409 
05*32309 
00*01 041 

72.29750 

70.2977C 

00*00933 

05*00049 

91*02273 


0*0 

0*12906 
0.29526 
0*39277 
0.54214 
0.76730 
0*09251 
1*33079 
1 *91491 
2*09903 
3.59079 
4*70900 
0*27625 
0*12232 
10.40500 
13.23125 
10.74236 
21.14509 
20*75177 
30*14703 
34*04014 
30*50596 
43*05203 
50*11000 
57*04000 
50*00706 
72.29935 
70*41015 
05*35007 
93*29901 
102.47900 
113*21143 
129*92027 
141 *23239 
190*11004 
100.01914 
171.10440 
103*01000 
197.04025 
214*17300 
233.09937 
255.30130 
207.90009 
201*71110 
290*00770 
313.09430 
332.27939 
353.07006 
370*40950 
402*95752 


Toblo  3.1.  Motriz  Ricottl  Solutlan  for  2-130 
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k*.. ...... 4^*.-  * 


(3.5-J^) 


lOOv  -f  Xg  -  0 


(3.5-5) 


Costate  equations  are 


1“  “*1 


Xg-  -X^  -  *2 


+1  if  Xg  <  0 

1  if  Xg  >  0 


(3.5-6) 

(3.5-7) 

(3.5-8) 

(3.5-9) 


Ug  ■  Xi  If  Xg  ■  0 


along  the  singular  sure  the  stregnthened  GIC  condition 


(3.5-10) 


Is  satisfied,  and  transversallty  conditions  are 


Xl(3.1T)  -  0  (3.5-11) 

X2(3.17)  -  0  (3.5-12) 

In  this  exBople  a  sequence  of  control  >  -1  for  0  ^  t  ^  t^ 
and  u  ■  -JL  for  t  <  t  <  3. 17  with  one  switching  from  aon-slngulai* 

S  ^  8  ** 
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control  to  singular  control  satisfied  all  the  necessary  and  trans- 

versality  conditions  for  the  opfclaality.  To  integrate  the  set  of  state 

and  costate  equations  (2.6-Ul*.)  -  (2.6-l<-7)  with  boundary  conditions 

(2,6-3) »  (2.6-29),  (2.6-37)  and  (2.6-38),  by  the  first  approetch  we 

used  a  computer  program  in  SSP  subroutine  LEWT  for  TSWP  and  subroutine 

HPCL  for  initial  value  problems.  For  the  initial  guess  t  ■  .8 

s 

after  six  iterations  the  problem  stopped  with 

lx  »  6  X  10"^  <  10“^  (3.5-I3) 

the  optimal  switching  time, 

tg-i.l950»i6  ~1.2  (3.5-11^) 

and  the  performance  index. 


J  -  .60243  (3.5-15) 

Table  3,2  shows  Iterative  and  computer  results  for  this  problem. 
Tables  3.3-a,b)  show  comi^ter  printout  of  the  state  and  costate  tra¬ 
jectories  on  singular  and  non-singular  intervals.  Figures  3. 4-3,7 
illustrate  the  behavior  of  the  controls  and  trajectories.  The  com- 
pitational  time  at  each  iteration  with  IBI  37O  was  approximately 
•54  second. 


Iteration  No. 

1 

Switching  Time 

Terminal  Error 

h 

1 

.8 

56.058021 

9.137982 

2 

.973001 

23.236232 

2.370761* 

3 

1.085023 

9.150887 

.515614 

k 

1.15131^2 

3.262189 

.987125 

5 

1.188061 

.667985 

.005082 

6 

1.19501*6 

6  X  10"^ 

Tatile  3,2.  Computation  of  Switching  Time  t^ 

s 


Saddle  Point  Solution 


Time 

X 

1 

”-0.00909 

8.59047 

0.1 0080 

0.65424 

C.20009 

0.700 1 1 

-0.30090 

0.73696 

0.40090 

0.762-9 

o.seoo*' 

e. 7761 7 

0.60000 

0.76429 

_ 0.70090 

C. 78945 

0.80900 

0.76662 

0.99090 

0.742  61 

0.95000 

0.72716 

.  j.eeeee 

0.70991 

.1  .05900 

-.68636 

i.iecoc 

8.66529 

1.15093 

0.63955 

— l-20eC9 

0.61141 

X 

2 

<*•60706 

c.sossr 

0.4C9IS 
e. 30993 
C««I0SS 
0.1  I l»3 

o.ei  I*** 
-0.08935 
-9.1681 9 
-0.28807 

-0.338C6 
-0.38892 
-0.43801 
-0. 48600 
-0.53890 
-C .58800 


1.45739 
1.39467 
1 .32687 
1  .25498 
I . I  7990 
1.  1029<' 
1 .02469 
0.94637 
0.86893 
9.79338 
0.75662 
0.72971 
0 .68576 
9.65191 
0.61928 
9 .58890 


1 .241 75 
1.0433? 
9.861 35 
C. 69627 
•‘.54647 
0.41824 
0.30573 
0.21193 
0.13419 
0.97482 

e.esi 72 

0. 03294 
0.91644 
0.00615 
0.00203 
0.9 


Table  3*3-6*  Numerical  Reeults  for  Kon<SinguIar  Subare 


Time 

*1 

*2 

V 

— 1.20CSC 

e.61141 

•0.5889 : 

8.58899 

0.9 

1 .250CC 

0.58276 

-8.55815 

0.55815 

0.9 

1 .30909 

0.55557 

-C. 52970 

0.52979 

0.7 

1.35909 

0.52977 

-0.59257 

0.59257 

0.0 

— i«aoeec 

8.50529 

--e.4767C 

0.47679 

0.9 

1.45000 

0.46298 

-0.45292 

0.45202 

9.- 

1 .50009 

0.46097 

-0.42647 

0. 42647 

0.0 

1 .69-99 

0.41946 

-0.38453 

8.38453 

0." 

_ 70890 

.  0.38394 

-0.34444 

^•34444 

8.9 

1 .0-800 

0.35945 

-0.30785 

C. 30789 

0.“ 

1 .909-0 

0.321 38 

-8.27423 

0.27423 

c  •  r 

2.00009 

0.29551 

-0.2434 1 

0.24341 

o.c 

-^looce 

•0.27261 

—  .^e«21503 

.  .8.21503 

.  C.9 

2.2-000 

0.25244 

-0 .18860 

0.18889 

r.- 

2.3000- 

0.23479 

-0.16446 

0.16446 

c.« 

2.4990- 

0.21940 

-0.14176 

0.14176 

9.9 

—2.58000 

-8.29639 

.  •*8.12049 

.  .  _8.12949 

0.9 

2.69009 

0.19535 

-9.1 0042 

C.  10042 

0." 

2.70980 

0.16627 

-9.08135 

0.08135 

0.0 

t.oro'io 

9.1 7995 

-0.06310 

0 .06319 

0.0 

_2. 98008 

.  8.17363 

--0.O4S4O 

_ 0.04548 

0.0 

3.-0009 

0.16994 

-9.02832 

0.02832 

e.o 

3. 09999 

0.16795 

-e.oi 144 

0 .0 1 144 

-  .0 

3. 19999 

0.16765 

0.09533 

-0.90533 

r 

Table  3.3-b.  Numerical  Reeults  for  Singular  Subarc 


CHAPTER  If 


CLOSE3).IOOP  STa^TEGT  DEPHHDENT  SOTUTIOR 
ABD  AFPIICATICXI  OF  INVEBSE  SYSXE21S  IN  SINGUIAR  FROBIEMS 

As  was  mentloaed  before,  in  differential  game  problems  closed- 
loop  solutions  are  more  desirable  than  open  loop  solations. 

In  a  closed  loop  solution  each  player's  strategy  is  based  upon 
Information  about  the  current  state  of  his  opponent  and  ass\imes  that 
the  opponent  plays  optimally.  In  the  case  where  one  player  plays  non- 
optimaUy,  the  opponent  might  be  able  to  perform  even  better  than  the 
closed  loop  solution  if  he  can  determine  the  nonoptimal  strategy  of 
the  other  player.  In  this  case  the  Imown  strategy  of  the  opponent  can 
be  asstimed  as  an  external  input  to  the  system  and  the  ^me  problem  is 
converted  to  a  one-sided  optimal  control  problem.  It  is  obvious  that 
the  performance  that  can  be  achieved  in  such  a  way  would  be  better  than 
the  performance  achievable  either  with  open  loop  or  closed  loop 
strategies.  The  following  static  example  will  Illustrate  how  the 
deviation  of  one  player  in  a  zero-sum  game  from  the  saddle  point  solu¬ 
tion  may  affect  the  payoff  of  the  game  and  how  the  opponent  can 
achieve  different  performances. 
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New,  assuioe  v  chooses  a  strategy  other  than  the  saddle  point 
*  ^ 

V  aO>e.g.  Vsl  and  u  follows  the  same  saddle  point  strategy 
* 

u  ■  0.  Then  the  value  of  the  game  will  be 

.  -1  (I'.l-S) 

We  note  that  u  has  achieved  a  better  performance  than  the  saddle 
point  case.  But  u  can  achieve  even  a  better  performance  than  (4-.  1-6) 
based  on  the  information  that  v  ■  1.  In  this  case  it  is  enough  to 
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minimize  (4.1-7} 


Thus 


so  that 


+  u  -  1 


-  2a  +  1  «  0 


u  « 


1 


(4.1-7) 


(4.1-8) 


(l^.l-9) 


Is  optimum  and 


J(u,v)  «  -1.25  <  -1  (4.1-10) 

Consequently  u  has  performed  better  In  comparison  with  the  case 
that  be  used  his  saddle  point  strategy. 

This  example  shows  even  a  reasonable  approximate  kaowledge  of  v's 
strategy,  can  help  u  to  do  better  than  his  saddle  point  strategy. 

For  example  if  y  «  1  and  u  has  an  approximate  estimate  of  v, 
e.g.,  y  m  .9,  then,  going  throu^  the  steps  before  to  determine  the 
optimal  u. 


-  11^  +  .9u  -  ,8X 

(4.1-11) 

25* 

(4,1-12) 

u  •  -.45 

(4.1-13) 
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J(ff,v)  .-1.D125  <  -1 


(Iv.l-U) 


So,  u  Is  still  better  off  following  this  control  rather  that 
his  saddle  point  strategy. 

Remark.  Ih  general,  samatlmes  a  player  may  not  be  able  to  aohieve 
better  than  his  saddle  point.  ?or  example  consider  the  case 

J  -  u  -  (4.1-15) 

|u|  <  1  (4,1-16) 

where  u  is  minimizing  and  y  Is  maximizing  j.  The  saddle  point 
solution  is 

u*  -  -1  V*  -  0  (4.1-17) 

If  V  deviates  from  the  saddle  point  strategy  the  best  strategy 
for  u  is  to  play  the  saddle  point  strategy  u  *  -1,  for  any  deviation 
of  V.  This  case  may  occur  for  dynamic  problems  in  the  case  that  con¬ 
trol  u(t)  is  always  on  one  boundary  of  his  control  region.  But  in 
cases  of  bang  bang  and  totally  and  partially  singular  u(t)  deviations 

of  v(t)  can  effect  switching  time  and  seme  change  in  singular  control 

arc  and  even  it  may  change  the  whole  sequence  of  singular  and  non- 
singular  u(t). 

In  pursuit-evasion  game  i^oblems  some  methods  have  been  suggested 
to  determine  the  opponent's  strategy.  These  approaches  may  be 
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(47) 

categorized  as  estimation  techniques^  *  or  inverse  system  tech*^ 
niques^^^^*^**^\  In  the  past  singularities  have  been  excluded  ftom 
these  problems .  In  this  chapter  we  will  utilize  the  concept  of  inverse 
systems  (assuming  its  existence)  to  determine  the  opponent's  control 
throu^  a  state  or  output  measurement.  Thus,  together  with  the  pro¬ 
posed  techniques  of  Chapter  3,  enables  us  to  generate  an  approximate 
closed  loop  strategy  dependent  solutlom.  In  the  next  section  we  will 
discuss  the  concept  of  Inverse  systems,  and  their  existence,  and  then 
we  will  shew  how  by  such  a  system  to  determine  the  strategy  of  the 
opponent. 


Inverse  S3 


4.1.a  Basic  Definition.  Let  U  and  U  sets.  A  mapping  S:U  ■* 
is  said  to  be  Invertible  if  there  exists  a  mapping  S:  V  -»  U  such  that 
^  and  SS  are  identity  mappings  on  the  sets  U  and  V  respectively. 

A 

In  this  case  3  is  said  to  be  an  Inverse  of  S.  Since  such  an  inverse 
if  it  exists,  is  tinlque,  we  will  denote  the  inverse  of  3  by  3^, 

The  following  deflniticxi  is  due  to  Zadeh  and  Pessoer  in 
Reference  ( 3) . 


A 

4,l.b.  Formal  Definition.  Let  g  and  g  be  characterized  by  in^t- 
output -state  relations  of  the  form 

g  :  y  «  I(x;u)  *  ®  (4,2-1) 

g  ;  w  -  J(z;v)  z  e  2^ 


(4.2-2) 


ft 

where  u  and  v  are  Inputs  to  S  and  3  respectively,  and  y  and 
w  represent  the  corresponding  outputs,  z  and  z  are  corresponding 
states  of  each  system  (the  output  function  space  of  S  is  asstuned  to 
he  the  input  function  space  of  S  and  conversely).  Then  g  is  called 

A 

inverse  to  S  or  S  is  c  ailed  inverse  to  g  if  and  only  if  to  every 

A 

state  z  of  g  there  ezlsts  a  state  z^  of  g  such  that 

JCZjj*  I(3c;u))  -  u  Vu  (1»-.2-3) 

A 

and  conversely  to  every  state  z  of  g  there  corresponds  a  state  z 

z 

of  g  such  that 


I(z  ;  J(z,v))  «  V  Vv  (l*’.2-4) 

z 

If  g  is  inverse  to  g  then  g  is  denoted  by  and  g  de¬ 

noted  by  g“^.  Correspondingly,  the  state  z^  is  denoted  by  z"^.  g 
will  be  said  to  be  invertible  if  it  has  em  inverse. 

From  this  definition  it  follows  that  if  {h,2-3)  and  (14-.2-4)  are 

satisfied  with  states  z,z_,  and  z,z  respectively,  then  they  eire 

X  z 

also  satisfied  with  the  states  z  ,z  and  z_,z. 

z  z 

For  this  study  we  will  Include  theorems,  without  proof,  and  in  the 
following  we  also  include  several  pertinent  definitions.  For  more 
detailed  study  of  this  subject  references  (3)  and  (10)  are  recoomended. 

Theorem  It-.l.  A  mapping  g  :  U  -  V  is  Invertible  if  and  only  if  it  is 
one-to-one  and  onto. 
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T.«wmnft  U.i,  If  a  mapping  8  :  U  -*  4  is  Invertible  then  8*^  is 
invertible  and  (8"^)”^  ■  8. 

Definition  k,2,  A  mapping  8  :  U  -  V  is  said  to  be 

a)  Pra-irsertible  or  left  invertible  if  a  mapping  8  ;  ft(  S)  -*  U  exists 
such  that  88  •  where  a(  8)  denotes  the  range  of  8.  In  such  a 
case  8  is  called  a  left  or  Bbst-lnvertible  8  amd  denoted  by  8£^. 

b)  Postlnvertible  or  ri^t  invertible  if  a  mapping  8  ;  V  -•  U  exists 

such  that  ^  In  such  a  case  8  is  called  a  ri^t  or  post- 

Inverse  of  8  and  denoted  by  8^^. 

Theorem  4.2.  If  8  :  U  -  V  is  both  left  and  right  invertible  then  8 
is  Invertible  and  8*^  ■  8^^  «  8^^. 

In  the  remainder  of  this  chapter  we  will  consider  only  linear 
dynamical  systems  indicated  by  the  notation 


8  s  (A,B,C,D) 


(4.2-5) 


This  notation  specifies  the  following  set  of  state  euid  output 
equations . 


x(t)  ■  Ax(t)  +  Bu(t)  (4.2-6) 

y(t)  ■  Cx(t)  +  Du(t)  (4,2-7) 


where  x  e  r“,  u  c  y  c  are  the  state  input  and  output  vectors 
respectively.  The  matrices  A,  B,  C  and  D  are  real,  in  general  time 


-rarylng,  coatinuous  and  of  ccopatlbla  sizes.  The  initial  eonditiois 
are  assumed  to  be  zero. 

Ve  now  address  ourselves  to  the  issue  of  the  existence  of  inverse 
systesm  for  (l4-.2-5)»  and  the  caastaructlon  of  such  a  system  if  Indeed 
they  do  exist.  Before  ve  discuss  conditions  for  the  existence  of  such 
Inverse  systems  and  the  algorithm  to  construct  them,  ve  briefly  reviev 
the  applications  and  the  literature  of  this  subject. 

The  concept  of  inverse  systems  has  found  applications  in  numerous 
problems  of  engineering.  Information  recovery,  in  coding  theory  is 
one  of  the  areas  in  which  Inverse  systems  are  used.  A  linear  time 
invariant  dynamical  system  can  work  as  an  encoder  for  a  special  type 
of  code  and  the  post  inverse  of  the  system  is  used  as  a  decoder. 

A  post  inverse  system  has  been  used  in  filtering  and  estimation 
theory  in  the  presence  of  colored  noise.  It  is  used  to  whiten  the 
colored  noise  which  is  easier  to  analyze. 

In  atochastlc  differential  game  problems  the  pursuer  typically 
attempts  to  estimate  the  state  of  the  evader's  system,  and  then  eoploys 
an  Inverse  system  to  determine  the  evader's  input. 

In  the  deterministic  differential  games  inverse  systems  have  been 
used  to  determine  the  opponent's  strategies  from  the  perfect  measurement 
of  the  state  of  the  system  in  non -singular  problems.' 

Other  applications  Include  decoupling  of  multivariable  systm&s, 
network  synthesis,  network  realization  of  passive  impedances,  and  two 
point  boundary  value  problems. 

In  the  present  work  we  apply  this  concept  to  develop  a  solut-'on  to 
deterministic  linear  differential  game  with  singularities. 
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Most  research  vhlch  has  been  done  in  inverse  systems  is  concerned 
with  time  Invariant  systems.  Seme  authors  have  proposed  algorithms 
vhich  may  be  used  for  special  classes  of  differential  game  problems. 
Our  need  is  for  some  criteria  for  invertibiUty  of  the  system  and  also 
cm  efficient  algorithm  for  eonstmetiag  the 'inverse  systepi. 

Patal^^^^  (1973)  has  obtained  a  sufficient  condition  for  Invert- 
iblllty  of  a  special  class  of  time  invariant  systems  in  which  m  i. 
This  criterion  simply  tests  the  rank  of  the  product  of  two  matrices. 

One  of  the  most  recent  works  for  time  invariant  systems  in  the 
case  of  i  m  is  the  work  of  Slnwat  and  Pallside^^^  (1976).  They 
have  proposed  an  algorithm  which  is  based  on  the  factorization  of  the 
transfer  matrix  of  the  system.  The  criterion  for  invertibllity  here, 
requires  the  formation  of  the  transfer  matrix  and  determinatlcxi  of  its 
rank.  Pull  rank  of  the  transfer  matrix  is  a  necessary  and  sufficient 
condition  for  the  applicability  of  their  algorithm, 

Silverman Introduced  a  finite  sequential  algorithm  for  time 
invariant  systems  and  later  on  he  extended  the  algorithm  to  the  time 
varying  systems.  Also  a  sequential  test  of  existence  is  incorporated 
in  the  algorithm. 

lt-,2-1  Inversion  of  Linear  Time  Invariant  Systems 

With  regard  to  the  basic  definition  of  the  Inverse  system  let 
R[S]‘^  be  a  ring  of  polynomial  matrices.  The  linear  time  Invariant 
system  (4.2-6)  and  (4.2-7)  whose  transfer  matrix  function  is 
a(S)  c  RCS]'^  and  which  is  assumed  to  have  full  rsink,  is  said  to  be 
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left  Inyertlble  if  £  >  m  and  there  exists  a  system  with  transfer 
function  matrix  Grj^(S)  e  R[Sl^^  such' that  Gj^(S)G(S)  »  is 

called  left  Inverse  of  G(S).  Similarly  the  system  is  said  to  be 
ri^t  invertible  if  £  <m  euid  there  exists  a  e  RCS]”^  such 

that  G(S)Gj^(S)  ;■  0^(3)  is  called  the  right  inverse  of  G(S). 

When  £  «  m  the  ri^t  and  left  inverses  atre  identical. 

The  foUoving  theorems  which  have  been  presented  in  references 
(59)  and  (50)  give  some  conditions  for  invertibility  of  linear  time 
invariant  systems. 

Theorem  lv.3.  A  sufficient  coiA^ition  for  invertibility  of  the  linear 
time  invariant  systems  (AyB,C)  is  that  '  rabk(CB)  ■  mln(£ym). 

Theorem  4.4.  A  necessary  and  sufficient  condition  for  invertibility 
of  the  linear,  time  invariant  system  (A,B,C,D)  is  that  G(S)  has  a 
full  rank. 

4.2.2  Inversion  of  Linear  Time  Varying  Systems  -  Regular  Systems 

Roughly  speaking  a  regular  system  is  a  system  in  which  P(t)  has 
a  constant  rank. 

Given  a  set  of  regular  linear  time  varying  system 


3: 


x(t)  .  A(t)x(t)  +  B(t)u(t) 
y(t)  -  C(t)x(t)  +D(t)u(t) 


(4.2-8) 

(4.2-9) 


for  each  given  initial  condition  z(tQ)  »  x^,  Equations  (4.2-8)  and 
(4.2-9)  define  a  mapping  H*  :  U  ->  y.  The  following  theorem  applies 
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to  this  situation. 


!nieorem  U-.3.  H_  :  U  -•  U  has  a  continuous  left  inverse  if  and  only  if 

-  Xq 

d_  ■  rank  D(t)  «  m  for  V  t.  In  this  case  H  has  a  unique  left 
U  *0 

inverse  denoted  by  So  the  unique  left  Inverse  to  the  system 

(h,2-8)  and  (h.2-9)  is  given  by 


: 


x(t)  =  [A(t)-B(t)D^(t)C(t)]x(t)  +  B(t)D^(t)y(t) 


u(t)  «  D’^(t)y{t)  -  D^(t)C(t)x(t) 


xC^q)  »  3Cq  (4.2-10) 


(4.2-11) 


vrtiere  «  (D'^(t)D(t))“-^  D"-(t)  and  m  <  ^. 


-1 


Theorem  4.4.  H_  :U-*U  has  at  least  one  cmtinuous  rirfat  inverse 
-  Xq 

if  and  only  if  =  rank  D(t)  «  I  for  V  t,  and  it  is  denoted  by 


If  d^  »  rank  D(t)  ^  I  for  some  t  e  Then  does 


not  have  a  right  inverse  (continuous  or  not). 


Theorem  4.5.  H  :  U  -  4  lias  a  unique  continuous  inverse  if  and  only 

-  Xq 

if  dQ  «  rank  D(t)  e  m  »  jC  for  V  t.  The  inverse  of  the  system 
(4,2-8)  and  (4.2-9)  in  the  case  that  D(t)  is  a  square  matrix  with 
full  constant  rank  is  expressed  as: 


86 


(Iv.2-13) 


f  i(t)  =  [A(t)-B(t)D‘^(t)C(t)]x(t)  +  B(t)D“^(t)y(t) 


x(to)  «  Xq 


D'^t)y(t)  -  D‘^(t)C(t)x(t) 


V- 

In  cases  above,  D(t}  had  full  and  constant  rank  m  or  I,  Novr 
we  consider  cases  that  m  x  m  matrix  D(t)  has  constant  rank  but 
d^  <  m  or  the  case  where  i  x  m  matrix  D(t)  has  dQ  =  rank  D(t) 
<  mln(  i,m) . 


1|.,2,3  Inversion  Algorithm  for  Construction  of  the  Inverse  System 

The  basis  of  the  Silverman’s  algorithm^ exploits  the  following 
theorems  by  Dolezal, 

If  D(t)  is  a  matrix  with  a  constant  rank  dQ  <  min(m,i)  on 
[tQ,t^]  and  differentiable  then  there  exists  a  square  non-singular 
liatrix  SQ(t)  such  that 


SQ(t)D(t) 


(iv.2-15) 


where  Sg(t)  is  m  x  m,  m  <  /  and  DgCt)  has  dg  rows  and 
rank  DQ(t)  -  dg  on  [tg,t^3. 

Defining  a  system  as 


^x(t)  •  A(t)x(t)  +  B(t)u(t)  (l^•.2-l6) 

yg(t)  -  Cg(t)x(t)  +DQ(t)u(t)  (1^.2-17) 
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( 


where 


yo(t) 


SQ(t)y(t) 


■  yo(*>  ‘ 

■CoW 

m 

yQ(t)  ^ 

.  CnM  -Sg(t)Qt)  . 

U  VJ 

Jt(t) 

(1^.2-2£) 


The  her  and  the  tilde  represent  respectively  the  first  rows 
and  the  last  m-d^  rows  of  the  matrices  CQ(t)  and  yQ(t). 

Now  define  a  matrix  differential  operator  Mq 


(1^.2-19) 


where  Id^  is  an  identity  matrix  with  order  d^.  Then 


■  yo'*)  ■ 

ffoCt) 

5o'*>  ■ 

Moyo(t)  - 

d  ^  ^ \ 

x(t)  + 

_CfQ(t)B(t). 

(4.2-20) 

Consider 


rank 


■  ■ 


(lt.2-21) 


if  d^  ■  m,  then  the  inverse  it  found  as  before.  If  d^  <  m  then  we 
find  S^(t)  such  that 
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Vk- 


- - 1 

cf 

■  C^t)  ■ 

d 

• 

x(t)  + 

L 

.  (f^(t)B(t)  . 

u(t) 


(U.2-27) 


where 


-  e^(t)A(t)  -  k(t) 


(lf.2-28) 


Now  if 


rank 


Le'v(t)B(t)  J 


<  m  and  constant  for 


t  c  [t^jt^].  Then  an  m  x  m  non -singular  differentiable  s  ^(t) 

iC  • 

be  found  such  that: 


ffk(t)  ■ 

■B 

■  c-^ct)  ■ 

ff^(t)B(t) 

0 

m  m 

(lf.2-29) 


where  has  d^^^  rows  and  rank  .  d^_^^  and  then  the 


system  is  defined  as 


\+l* 


1 


x(t)  .  A(t)x(t)  +  B(t)u(t)  (1^.2-30) 

yk+l^*^  “  (1^.2-31) 


where 


y^+ift)  .  W*  Vk<*> 


(1^.2.32) 
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(4.2-33) 


Also  5^+i(4)  liv^,  rmn  and  las  si-dv+i  rows. 

Suppose  noMT  that  there  exists  an  Integer  a  ^  such  that  has 
ranlc  m.  Then  it  Is  possible  to  get  the  inverse  in  the  form 


x(t)  •  [A(t)  -  B(t)D^(t)C(t)]x(t)  +  B(t)Dj^(t)yJt)  x(tQ)«XQ 


u(t)  -  D®(t)y(t)  -aD^(t)C(t)x(t) 


(1^.2.35) 


where 


D^Ct)  «  (Dj(t)DQj(t))’^J(t)  and  y^^Ct)  •  (  M  • 

LbO 

If  i<m,  then,  Dj(t)  -  Dj(t)(Dj^(t)Dj(t))"^  and  if  f-m  the 
Inverse  system  is  found  as  the  form  of  (4.2>13)  and  (U.2-llt-). 

Now,  a  more  precise  definition  of  the  regular  system  is  the  case 

mm 

when  matrices  D(t)  and  (8|^(‘t:)B(t)  ]  have  constant  rank  on 

t  6  [tQ,t^3  for  h  w  0,1, ••  0,0*10 

So  it  is  concluded  that  the  regular  system  representation  (4o2.8) 
and  (lt.o2*9)  is  invertible  if  there  exists  a  positive  integer  a  <  n 
sudi  that  d^  ■  m  ■  /o  In  the  ease  d^  •  m  <  X  the  system  is  called 
left  invertible  and  in  the  case  d^  ■  X  <  m  it  is  called  right 
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inrertlblB.  Refereaeea  (5^  (^)  shov  the  necessity  proof  of  the 

abonre  results. 


4,3  Determination  of  Opponent  Strategy 

In  this  section  «e  use  the  results  of  previous  sections  to  verify 
the  invextiblllty  of  the  system  (2.6-2)  and  If  It  exists  to  ccmstruct 
it  and  determine  control  ▼.  We  assume  each  player  has  perfect  infor¬ 
mation  about  the  output  or  the  state  of  the  system. 

Let  u  ,  V  and  z  be  the  sctddle  point  solution  of  the  differ¬ 
ential  gsme  (2.7-1)  to  (2.7-4)  then  we  have 


Z* (t)  »  A(t)z* (t)  +  B(t)u*(t)  +  C(t)v* (t) 

(‘^.3-1) 

y  (t)  -  z  (t) 

(‘^.3-2) 

It 

M 

(i^.3-3) 

If  ^(t)  is  the  deviation  of  the  saddle  point  state  trajectory 
due  to  the  deviation  of  the  evader's  control  Av(t)  and  ^(t)  is  the 
change  of  pursuer's  control  in  response  to  ^(t),  we  will  have 


v(t) 

-  r  (t) 

+  ^(t) 

u(t) 

■  u*(t) 

+  £w(t) 

x(t) 

»  X* (t) 

+  ^(t) 

y(t) 

-  y*(t) 

+  ^(t) 

and  (4. 

,3-4)  we 

get 

^  V  t  e  [tQ,t^]  (4.3-4) 
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^(t)  m  A(t)^(t)  +  B(t)iJu(t)  c(t)£w(t)  (^.3-5) 

Ay(t)  •  iCac(t)  (^.3-6) 

^(^q)  -  0  (><-.3-7) 

We  assuaie  the  output  neasurenent  is  taken  by  the  pursuer  period¬ 
ically  in  a  small  time  interval  At  m  t^^^  -  t^^  1  «  0,1,,,,  ,  When 

the  game  starts  running,  the  pursuer  takes  a  measurement  at  *  *^0 
At  and  ccBipares  this  value  vith  the  saddle  point  solution  and  notices 
the  difference  ^(t^)  •  ^(t^)  ehidi  is  caused  by  non-optimal  playing 
of  the  evader's  strategy.  Since  at  this  time  the  pursuer  knoirs  his 
om  strategy 


Au(t)  -0  at  t  ■  tj^  (1*‘.3-6) 

so  at  any  time  of  the  measurement 

Aac(t)  .  A(t)Ax(t)  +  C(t)Av(t)  (lv.3-7) 

AxCtp)  .  0  (1<-.3-8) 

Ay(t)  .  Ax(t)  (l»-.3-9) 

Assuming  Invertibllity  criteria  of  the  Section  k,2,  i,e,,  Theorem 
14-.3  ^or  the  linear  system  (k,3~7)  "  (^•3'‘9}  hold.  Since  there  is  per¬ 
fect  infonnatioa,  by  measuring  the  output  and  by  knowing  the  saddle 
point  at  each  time  ^(t^)  is  conputahle  by  the  pursuer.  Having  a 
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sufficient  history  of  observntiOT  time  deriTstives  of  ^(t)  can  be 
conputed  at  each  time  by  forward  difference  appraadmatian.  Hence 
iSsv^t^)  is  determined  at  each  instant  of  measurement  using  (U.2-3lv) 
and  (^*2-35)  •  So>  st  this  time  it  is  assumed  that  the  evader  continues 
the  gsme  with  the  same  deviation  f^om  the  saddle  point  trajectory  until 
the  end  of  the  game. 

▼(t)  -v^Ct)  +dsv(tj^)  for  Vt  e  [t,t^]  (14..3-10) 

How  in  this  time  Interval  v(t)  is  considered  ais  an  external 
input  to  the  system  and  a  one-sided  singular  optimal  control  problem 
vith  initial  condition 

x(t^)  .  x*(tj^)  4  dacit^)  (‘v.3-11) 

is  solved  by  applying  the  proposed  technique  for  singular  problems. 

The  control  u  is  obtained  periodically  at  each  time.  Again  at  time 
tg  ■  t^  4-  dt  another  measurement  is  taken  and  coeipared  with  the  recent 
updated  trajectory  and  procedure  is  continued  until  the  game  is  ter¬ 
minated.  Hy  this  technique  a  suboptimal  solution  can  be  obtained. 

If  the  evader's  deviation  from  the  saddle  point  solutlcm  is 
sufficiently  small  then  the  pursuer's  sequence  of  singular  and  non- 
singular  control  will  be  the  same,  m  this  case  the  location  of  switch¬ 
ing  time  and  the  value  of  singular  control  will  be  changed.  If  the 

deviation  is  large  enough  the  sequence  of  singular  and  non-singular 
« 

controls  will  be  changed. 

. 


Ita  non-UiuMr  problms  the  systen  equations  are  linearized 
arooad  a  known  refemee  traject<ny  (usually  saddle  point  trajectory). 


X  m  f(at,u,v) 


(4.3-14) 


*(  V  *  *0 


(V.3-15) 


y(t)  -  x(t) 


(4.3-16) 


Assuming  f(x,u,v)  Is  continuous  in  x,  u  and  v  the  linearized 


equations  are 


6x  m  f  ax  +  f  ,8iu  +  f  8v 
X  u  y 


(4.3-17) 


8*(tQ)  -  0 


(4.3-lB) 


where 


^x“Sx*  »  ^u-SS-  *  ^t“57U 


(4.3-19) 


Taking  measurement  of  the  output  at  seme  time  Interval  At  «  t 
-  t^,  1  «  0,1,...  and  approximating 


ax  «  X 


8u  ■  u 


real 

real 


ar  •  V 


V  t  €  ttQ,t^3 


real  vet 
-  yreal  -  ^ref 


(4.3-20) 
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sinllftr  to  th«  linear  case  at  each  tine  of  the  measorement 


Bu  •  0 


therefosre 


8x  >  f  8x  +  f  8v 

X  ▼ 

^  8x(tQ)  .  0 

Sy  >  8x 


(4.3-22) 

(*^.3-23) 

(4.3-24) 


Assuming  the  inverse  to  (4.3-22)  -  (4.3-24)  exists  the  procedure  will 
he  similar  to  linear  case. 

Note;  This  method  requires  rellnearlzatlon  and  it  Is  assumed  that  the 

deviation  Is  slight  and  shairp  deviation  does  not  occur  frequently. 

Althouj^  this  does  not  restrict  the  method  hut  practically  reduces  the 

accuracy  of  the  technique. 
t  liQ  hi) 

Anderson^  ’  '  has  proposed  a  near  optimal  method  hy  taldng 

measurement  of  the  states  at  eipially  time  interval  At  and  updating 

the  set  of  TPBVP  obtained  from  the  necessary  conditions  for  the  saddle 

point.  But  since  the  solutions  obtained  hy  this  procedure  sure  based  on 

the  assumption  that  both  players  play  optimally  the  solution  cannot 

he  as  good  as  the  case  that  one  player  knows  the  opponent  strategy. 
f47) 

Jachlnovitz'  used  estimation  techniques  to  determine  the  evader* 
control  throufi^  state  measurement  and  achieved  a  better  performance  than 
Anderson.  But,  his  ^put  estimatimx  algorithm  was  very  time  ccmsumlng 
so  that  it  was  impractical  for  on  line  purposes.  By  Inverse  system 
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t«ehniqae  inpat  datennlnatioa  is  dme  very  fast  so  It  is  convenient 
for  on  line  use. 

Iv.lt-  On  Line  Solution  of  the  Example 

Wie  consider  the  numerical  example  In  Section  3.5.  Suppose  the 
evader  due  to  the  biased  error  In  his  ccntrol  has  some  deviation  ftaa. 
the  saddle  point  trajectory.  Assume  this  deviation  Is  Av  >  .1  su<^ 
that  v(t)  m  V* (t)  +  .1  for  0  <  t  <  3«17  where  v* (t)  Is  his  saddle 
point  solution.  This  control  v(t)  Is  not  Imoim  to  the  pursuer  xmless 
he  determines  It  by  direct  observation  and  measurement  of  the  output  or 
state  of  the  system.  In  this  exaaqple  we  assume  there  are  perfect 
measurements  and  y(t)  >  z(t). 

At  time  t  «  0  the  system  starts  running  in  real  time.  Some 
mesksurements  are  taken  at  some  small  time  intervals  and  compared  with 
the  saddle  point  trajectories.  Consider  the  system  (14-.3-T)  >  (^•3‘*9)» 
since  rank  [C]  «  1  for  this  example,  from  Theorem  ^.3  the  system  Is 
exists.  So,  the  evader's  Input  deviation  ^(t)  is  found  from  the 
following  relationship: 

^(t)  .  (C®C)“Vdac(t)  -  (If.Wl) 

dy(t)  B  dx(t) 

In  the  following  we  find  the  difference  between  the  saddle  point 
trajectory  (starred  quantities)  and  real  trajectory. 


and  Av(*l)  ■  0.102.  Taking  another  measurement  at  time  t  b  0.2  we 
will  get 


and  Av(.2)  b  .1020.  It  Is  noticed  that  the  computed  deviation  Is  .102 
from  the  optimal  open  loop  strategy.  Therefore,  using  the  external 
control  v(t)  b  v  (t)  +  .102  over  *2  <  t  <  3.17  with  the  initial  con¬ 
dition  x^(.2)  B  .70lt'20  «u2d  •  ,k2^36  a  one-sided  optimal 

control  problem  results.  This  problem  was  solved  and  the  pursuer's 
control  was  computed  as 


u  ■  -1  for  0  <  t  <  1,33 


""  "  ""singular  ^*33  <  *  <  3-17 

"singular  “  *1^*^  *  “  *1^^^  " 

Tables  (Iv.l^)  and  (4.1-b)  shov  the  computer  results  of  the  computa¬ 
tions. 

As  long  as  measurements  shew  that  Av  ~  .1  we  may  consider  this  to 
be  the  optimal  solution. 

Now  in  order  to  show  the  advantage  of  the  method  we  compare  the 
results  of  the  following  3  cases. 

Case  1.  Two  players  play  optimal  open  loop  (saddle  point  strategies). 

Case  2.  The  evader  deviates  from  his  saddle  point  strategy  but  the  pur¬ 
suer  plays  his  optimal  open  loop  strategy. 

Case  3.  The  evader  deviates  from  his  saddle  point  strategy  and  the 
pursuer  plays  his  optimal  closed  loop  strategy. 

The  following  tables  show  the  results  of  the  computation  for  these 
cases . 


Av(t) 

*s 

J 

0 

1.2 

.60243 

Table  (^.1).  Case  1.  Two  players  play  optimal  open  loop  (saddle 
point) . 


-,86687 
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Table  (V.2)  Case  2.  The  evader  deviates  and  pursuer  plays  his  optimal 
open  loop. 


The  computer  printout  of  this  ease  for  Av(t)  b  .1  is  given  in 
Table  (U^.lt-a)  and  (l4-.lv-b)  and  Figures  (4.1-U.2)  show  the  controls  and 
trajectories. 

Far  the  Case  3  the  results  are 


value  of  singular  control  la  sll^tly  changed  (Figure  U.3^,b). 

As  the  results  In  Tables  (^.1)  and  (^.2)  show  In  Case  2,  the 
pursuer  will  achieve  a  better  performance  than  Case  1.  The  results 
in  Table  (4.3)  show  that  In  Case  3  the  pursuer  can  achieve  even  a 
better  performance  than  Case  2. 

We  can  also  consider  the  cases  that  the  evader's  deviation  does 
not  remain  constant ,  but  changes  at  seme  times.  Two  cases  are  con¬ 
sidered.  In  the  first  case  the  evader  deviates  Av(t)  ■  .1  for 
0  <  t  <  .605  and  then  frem  time  t  *  ,603  on  plays  saddle  point 
strategy  \uitll  the  game  terminates . 

In  the  second  case  Av  •  .1  for  0  <  t  <  .605  and  Av  •  ,03 

for  .605  <  <  3»17. 

In  the  following,  again  we  calculate  the  control  deviation  Av 
by  using  the  Inverse  system  technique.  For 

For  0  <  t  <  .605  we  already  computed  Av  and  In  this  interval 
Av  was  computed  at  the  Instant  of  time  which  was  constant  Au(t)  s  . 

At  time  t  B  .605  measurement  was  taken  and  ccmpeurlson  was  made 
with  the  new  saddle  point  reference  trajectory.  The  results  are 


and  In  the  next  measurement  we  get 


Tlaa 

X. 

X 

1 

2 

"f  .C - 

9.S98«7 

s.fr‘!T'*a 

P.'JSCCf 

c.e^rTi 

r.562ae 

e. lei^" 

e.654T« 

0.SI83* 

C«  1 

e.eT95« 

P. 47389 

■  "r.JCtCC 

9.4?938 

f 

9.Tr»4« 

94  38440 

*.Ta«6i 

<•.34029 

c.ao'js'* 

e.77t?? 

0.251*7 

■T.s^rsf 

e.T«;ae 

•>.  iei7'‘ 

o.apeec 

C.S979S 

P. 0721 8 

e.reoec 

t.aesre 

-C. 01746 

p.eseco 

-0. 1972* 

c.e'c:? 

A.rass* 

-0. 1970 3 

i.p^e^e 

p.mees 

-0.28*93 

i.i“?ee. 

K,TI€At 

-0. 37687 

1  .*?epp 

e.»S«27 

-9. 466N5 

Table  Ruaerieal.  results  for  Case  2.  (Birsuer  uses  ua) 


Tiae 

*1 

*2 

“i;2e?p?  ■ 

•  0^68427 

-0.46885 

I.259P* 

0.66181 

-*,43134 

l.39P'‘9 

0.64114 

-9,39571 

1.350** 

0.42222 

-9.36*68 

“i;4P?<je 

0.60501 

-*.3269- 

1.450*0 

9.58957 

-9.29391 

1.5000* 

e.57S5P 

-9.26471 

1.5500* 

0.56316 

-9.23465 

*T1 600*0 

*0.55197 

-9.29745 

I.650P0 

0.54242 

-9,17974 

1.79000 

0.53395 

-9. 15445 

I.75P00 

0*52699 

-0. 12876 

■  r.9*9P* 

0.52104 

-9.10521 

1 .85*0* 

0.51648 

-0.08131 

I .9C0C* 

0.51 287 

-0. 05933 

1.950*0 

0.51 *55 

-9.i37C  3 

:2.ceP0* 

0.5*913 

-0. 01 645 

2.05000 

0.59892 

0.09442 

2.1 900* 

9.50955 

0,92377 

2. 159P0 

0.51  1  3* 

0.04337 

“2  .•2*90  9  •" 

0.51388 

0.06162 

2.25*09 

0  .51  74  7 

0,08*99 

2. 3000* 

0.52185 

0.09736 

2.3500* 

0.52721 

0.11487 

■  2.400*0 

0.53331 

0.13131 

2.  450 'TO 

0.54034 

0*14793 

2.49999 

0.54809 

0. 16365 

2.54999 

0.55671 

0.17953 

59999 

•0.566*3 

9.1946'* 

2*64999 

0.5761 8 

«. 2*987 

2.69999 

9,58700 

0.22444 

2.74999 

9.59862 

0.23915 

*2  .T9999 

0.51091 

0.20331 

2. 84999 

9.62396 

9.26757 

2*89999 

0.63766 

0.28149 

2.94909 

9.65210 

0.29532 

■2.99999 

0*6671 8 

*.3089*- 

3*04'999 

0.68299 

*.32206 

3.09999 

9.69944 

0.33099 

3,14998 

0.71 659 

0. 34048 

■  3*19998 

0.73438 

0.36284 

Table  (b.lUb)  Ruaerical  results  for  Case  2  (Krsuer  uses  u^) 
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Clo«ea  Loop  Strategy  Dependent  Solutions 


•>c*«Toe; 
e.039fl? 
O.ltCf 
-®.23eec 
o.sao<^o 
e<  *3aco 
3.83«?C 
*  0.*39r» 
e.Tsore 
c.aaeo? 

0.9390 
~u?3ce^ 
I  .sorrr 

1 .1 

— I'.JSPSP 
t.zaocc 
l.33e'*C 


e.S53a> 
0.01629 
0.6*999 
• 9.T1*63 
O.TS*'er 
0. TTr3« 
M.79S36 
-0.89939 

o.aceee 
9,79597 
0.77769 
—0.75 '*8? 
9, 73492 
0,71 496 
0,69366 
-“0.67011 
0,6443C 
0.61625 


0.66099 
0.50740 
0.49177 
0.49295 
0.31375 
5.22449 
0.13506 
0,9*554 
-0.04412 
-6.13300 
-0.22372 
-e. 31362 
-9.35*59 
-0.40357 
-0.44056 
-=0.49355 
-9.53055 
-9.50355 


1.60729 
1 .54*71 
1 .40433 
1.41  503 
1  .34  17* 
I  .26524 
1 .10653 
1.10046 
1  .92595 
0.94557 
6.0671  3 
0.79063 
0.78  35* 
C .71727 
0.602*5 
-0.04794 
0.615*7 
0.50355 


1.60631 
1.30590 
I .10667 
-*>.99093 
0,61723 
0.65995 
0.51937 
0.39560 
0.20899 
0.1993* 
0.12655 
9.07055 
0.04675 
0,93194 
0.91737 
— 0;t>e767 
C.9C19I 
e  .9 


Teiile  (**.5-6)  HumerlcaX  reeults  »  Hoo-elngul»o  ore 


Tiae 

*1 

*8 

■  1.T3O00 

-  0.-81  625 

-0 ,50355 

1.30000 

0.5578.1 

-0.55345 

1.4390* 

0.500** 

-0.52474 

1 .46009 

0.53533 

-0.49734 

- 1  *-5300  0 

--  '  -0.81113 

-  ■»8*«7H9  - 

1 .56900 

0,46*20 

-9.44621 

1.63000 

0.46649 

-0.42235 

1.73000 

0.42652 

-0.37773 

-  -r  =83909 

- - 0.390*1 

—  -•6.33699  -  • 

1 .939"" 

0.359-2 

-0.29944 

2.03034 

0.339  53 

-0.26490 

2.13909 

0.30594 

-0.23316 

— r-.2369e 

— 8.70411 

—  -wo, 29369 

2.33399 

0. 2651 3 

-0.1 7625 

2.43609 

0.248*1 

-0.15957 

2.53004 

0,23497 

-C. 12640 

-T.-*3ceo 
2.73006 
2. *3999 
2.936nO 
-3.03000 
3. 1  369* 
3.22999 


-0.22348 

0.21423 

0.2C7I3 

0,29219 

—0*19999 

0.19*97 

9.19904 


9.10350 
9.001*3 
C. 96350 
C.94014 
9.02039 
3.00925 
0.91959 


-T  ,52355 
0.55345 
0,52474 
0.49734 
-9*471 19 
0.44621 
9.42235 
0.S7773 
"Ov 33690 
0.29944 
0. *2649  0 
0.23316 
-O .20369 
0,17625 
9.15957 
0.1264* 
-0.10359 
0.9SI63 
9.06056 
0.64014 
■0.92~9« 
0.0C025 
-0,01*59 


Toble  (U.5.»))  Pijoerie*!  results  -  singular  arc 
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.700  -  .80493 


.700 


t 

1 

8 

£sc^  m  .00022 

dXg  .  .00477 

t  «  .7 

00 

.  .00231 

dig  -  .05210 

and  ftron  (4.4-1)  Av(.7)  >  .052. 

By  taMng  another  measurement  at  time  t  •  .75  we  have 


t 

-  .750 

«  .80236 

1 

1 

. 

07661 

t 

-  .750 

m  ,Q02Sri 

Xg  -  -.06934 

and  the  deviation  in  evader's  control  Is  ^(.75)  m  .05. 

By  this  procedure  av  «  .05  Is  obtained  and  if  we  continue  taking 
measurement  and  calculating  £sr,  we  will  see  ^  ■  .05  for  the  rest 
of  the  Interval. 

Tables  (4.6)  and  (4.7)  show  the  performance  and  switching  times 
for  Case  2  and  Case  3  when  the  evader  changes  his  deviation  during  the 


Table  (4.6)  The  evader  deviates  and  the  pursuer  plays  ofptlnial 
open  loop 


Table  (4.7)  The  evader  deviates  and  the  pursuer  plays  optjjsal 
closed  loop  strategy  dependent 
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DISCUSSIGR,  CCNCZUSIGN  m>  RBCCMMQIIDAIIOR 


5.1  Discussion  and  Conclaslop 

This  dissertation  Is  mainly  eoncemed  with  singularity  In  differ- 
entlal  game  problems  with  linear  systems.  A  class  of  generalized 
pursuit -evasion  games  with  linear  state  equations  and  bounds  on  the 
control  in  which  Hamiltonian  is  linear  in  pursuer's  control  was  intro¬ 
duced.  This  problem  for  some  values  of  initial  condltlcms  and  final 
times  can  have  an  optimal  solution  with  singular  interval.  A  sufficient 
condition  for  the  existence  of  a  saddle  point  has  been  obtained.  One 
of  the  advantages  of  this  condition  is  that  an  extremal  value  for  each 
one  of  the  five  parameters  of  the  game,  i.e.,  t^,  S,  R,  Q  and  C  can 
be  detezmlned  throu^  the  inequality  (2.6-18)  (provided  that  other 
four  parameters  are  known)  to  guarantee  the  existence  of  a  unique 
saddle  point. 

Due  to  the  constraint  on  the  control,  in  general,  there  is  no 
analytical  solution  for  this  class  of  games.  Rapid  and  efficient 
numerical  techniques  are  required  to  solve  physical  and  practical 
problems  with  singular  arcs  ax;d  ccxitrols  with  discontinuities.  An 
indirect  numerical  method  was  proposed  here,  that  could  generate  an 
accurate  and  fast  solution  to  a  class  of  singular  problems  with  llnesor 
systems  in  which  Terminal  Error  Function  is  only  function  of  switching 
times.  In  this  method  the  sequence  of  singular  and  non-slnguIar  arcs 
is  estimated  by  physical  or  mathematical  insight  to  the  problem. 


Newtoa's  method  was  used  to  iterate  the  T.E.F.  on  the  switching 

• 

time.  ^  llnearizatlcxi  this  technique  was  extended  to  solve  the 
same  class  aS  problenis  with  non-linear  systems.  For  a  broader  class 
of  problems  with  linear  systems  in  which  T.E.F.  is  a  function  of 
both  switching  times  and  initial  costates  a  second  approaudi  was 
proposed.  In  this  approach  we  iterate  T.E.F.  on  both  initial 
costates  and  switching  times.  As  was  aonsidered  in  Chapter  3,  the 
singular  problems  with  order  of  higher  than  one,  and  also  problems 
with  bang  bang  controls  could  be  solved  through  the  second  approach. 

In  singular  problems  with  non-linear  systems,  since  the  Junction 
conditions  are  linearized  to  the  first  order,  they  may  not  be 
satisfied  precisely  at  the  same  time. 

Because  of  accuracy  and  rapid  convergence,  the  proposed 
technique  is  superior  to  seme  other  numerical  techniques  for  singu¬ 
lar  optimal  problems,  e.g..  Gradient  Method,  Epsilon  Method  or 
Quaslllnearlzatlon-Epsilon  Method.  Semetimes  the  approximate 
solutions  generated  by  some  of  these  methods  may  provide  information 
to  estimate  the  sequence  of  controls  and  switching  times. 

A  numerical  example  was  solved  using  Newton*  s  technique  and 
with  an  appropriate  initial  guess,  for  the  switching  time,  the  solu¬ 
tion  converged  after  six  iterations. 

By  applying  the  closed-loop  strategy  dependent  method  some  num¬ 
erical  solutions  were  obtained  for  the  same  example.  Bxmihe  numeric¬ 
al  results  we  considered  that  if  the  evader  deviates  from  his  optimal 
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open  loop  strategy,  the  pursuer  can  perform  better,  provided  he  uses 
closed-loop  strategy  dependent  policy.  For  applying  this  Idiid  of 
strategy  the  existence  and  stability  of  the  Inverse  system  Is  required. 
The  opponent's  strategy  at  each  Instant  of  time  can  be  rapidly  deter¬ 
mined  by  inverse  systems  so  that  It  Is  very  appropriate  for  on-line 
use.  In  this  kind  of  closed  loop  solution,  state  or  output  measure¬ 
ment  should  be  taken  periodically  in  order  to  generate  an  approximate 
solution.  If  the  measurement  interval  At  gets  smaller,  a  better  and 
more  accurate  solution  Is  obtained.  However,  as  At  gets  smaller  the 
computatloi  time  increases.  Some  considerations  should  be  taken  on  the 
choice  of  At  so  that  computational  barrier  Is  not  encountered.  This 
Interval  should  be  smaller  than  the  smallest  time  constant  of  the  sys¬ 
tem  and  also  should  be  larger  than  computation  time  at  each  period. 

5.2  Recommendations 

The  study  of  the  literature  in  singular  optimal  control  problems 
shows  that  still  much  reseeurch  needs  to  be  done  on  the  problems  of 
computation  of  optimal  singular  control.  The  cmalysls  of  the  Junction 
points  should  be  studied  further  and  some  efforts  axe  required  for 
locating  the  singular  arcs  and  obtaining  a  sufficient  condition  for 
peui:lally  singular  problems. 

A  general  and  accurate  method  for  solving  non-linear  problems  with 
all  orders  of  singularities  Is  not  yet  available. 

The  present  work  stimulates  further  development  for  computational 
research  in  optimal  singular  control  and  differential  game  problems. 


The  class  of  differential  games  in  whldi  singularity  may  occur 
for  hoth  pursuer’s  and  evader’s  control  is  an  interesting  topic  for 
computational  research. 

In  the  case  above  closed  loop  strategy  dependent  solution  is 
another  aspect  of  the  research  which  possibly  encounters  the  difficulty 
of  discontinuities  in  controls. 

The  extension  of  the  differential  dynamic  programming -Epsilon 
method  or  Quasllinearlzatlon-Epsilon  method  to  differential  game 
problems  seems  to  be  advantageous  to  overccme  the  difficulty  of  esti¬ 
mating  the  sequence  of  controls  and  initial  switching  times. 

The  extension  of  the  proposed  techniques  to  free  final  time 
problems  with  terminal  manifolds  is  suggested  for  further  research. 
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AFISKDIX  A 


A.l  Proof  of  Theorem  2.1 

Here  we  shiall  derive  GLC  necessary  condition  (-l)^[d^/dt^  H^] 
>  0  for  the  ease  when  q  «  1. 

Considering  (2.1-1)  to  (2.1-3)  neighboring  extremal  theory 
we  will  have 


&X  a  f  bx  4-  f  &U 

(A.1-1) 

X  U 

6\  a  -H^6x  +  5\  +  H^\u 

(A.1-2) 

Bx(tQ)  a  0 

(A.1-3) 

Bx(tp  . 

(a.l-l4) 

From  the  second  variation 


5^J  .  I  [5x\^5x]^  +  I  j  [6xV] 
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SC  XU 
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U  H 
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L  ux  uu  J 

m  a 

dt 


(A.1-5) 


subject  to: 


“  \x®*  *  \u^  ^^*0^  “  ° 

6\  ■  H  &x  -  H^,Su 

V.  XX.  XX  XU 


(A. 1-6) 
(A.1-7) 


From  (A.1-7)  it  is  obvious  that 


(A.1-8) 


I  j  [8\^  +  +  6x^^)6xdt  s  0 


iDategrating  the  first  term  hy  parts  we  obtain 


“f  t 

I  J  [-B\^6x  +  )8\^f^  +  +  Sx’^^)6x]dt  +  I  [sAx]^^  .  0 

trt  0 


(A.1-9) 


From  (A.1-1)  and  (A.l-l^) 


I  J  +  Bx’^i^Bxldt  +  I  [5x\^8x]^  .  0 

■fcn  -  f 


(A.l-IO 


Subtracting  Equation  (A.l-ID)  f^om  Equation  (A.  1-5)  and  simplifying 
gives 

*f 

S®J  .  IJ  iix\^  +  )6tt4t  (A  .l-U 

^0 

3y  differentiating  the  inside  of  the  parenthesis  and  substituting  from 
(A. 1-6)  and  (A.1-7)  we  will  get 

k  "*\a  *  '‘•Xu*  *  '‘•’’X’x* 

(A.1-12 


(5*Xa  *  'A.u  *  -  '“’’X'x  *  '“'"'^>u 


Sategratlng  (A.l-U)  lay  parts  together  with  (A.1-12)  we  get 

8®J  -  -  IJ  [6x®(fl^)^  + 

*0 

-  I  t  (6»\,  ♦  ®“\x)S^Uo 

(A.1-14) 


where 


*  J  Bu(t)dt 


(A.1-15) 


By  similar  Integration  by  parts  of  (A.I-II4-)  and  substituting  from. 
(A.1-12)  and  (A.1-13)  we  will  hare 

-  I  J  [Bx’^C'^)^  +  8u'^(H^)^]Bu2(t)dt 

*0 

■\U  ~uu'  -l^t„  2  '"u'x  '  '“U'X. 


(A.  1-16) 


* 


(A.l-17) 


We  know  in  a  non-singular  problem  >  0  which  is  a  convexity 


condition.  For  the  singular  case  where  H,  ■ 

”  u  uu 


special  varlatldn  as  the  following. 


0  we  consider  a 


Figure  A-1.  Special  variation  in  control  for  deriving  GZC  conditions 

du  (whldh  is  chosen  to  he  a  posltiva  impulse  followed  hy  a 
negative  la^luse) ,  &u^  and  Sug  ax9  shown  in  Figure  (A-l) .  We 

loiow  that  ^  is  independent  of  u,  so  ■  0,  and  as  a  result 

of  double  impulse  Cz  ■  5\  «  0,  trm  which  all  the  tenns  in  (A.1-16) 
involving  6z  and  0\  vanish.  We  take  constant  during  the 

period  of  Ou,  ftroa  the  Figure  (A-l)  it  is  obvious  that 


(a.1-18) 


J  (Su  6u2)at  <  0 
*0 

So,  in  order  to  assure  >  0  it  Is  necessary  that 

(H^)  <  0  (A.1-19) 

This  vas  for  the  ease  that  u  appears  explicitly  in  the  second 
tijoe  deriTative  of  For  eases  that  u  appeeurs  in  the  hi^er  order, 

the  procedure  of  the  derivation  of  the  necessary  conditions  conceptually 
is  the  same  (see  References  (20),  (31)). 

Proof  of  Theorem  2.2.  From  (2.1-18)  and  (2.1-19)  1^7  hypothesis  we  know 

that  a(t)  and  S(t)  are  ecxxtinuous  and  have  at  least  r  continuous 

derivatives  at  t. . 

s 

Let  e  be  a  small  non-zero  real  number  such  that  t„  -t*  e  is  a 

s 

point  on  the  non-singular  side  of  t^  and  -  e  is  a  point  on  the 
singular  side  of  t^.  Also,  the  limit  of  u^^^(tg  +  e)  and 
u^^^(t  -  e)  when  e  -*  0  are  u^^\t  )  and  u^^^t  )  respectively. 

S  XI  8  S  S 

Define 

«P  •  \  (A.2-1) 

If  K  ■  2q  r  then  will  be  the  Icvrest  order  derivative 

of  9  which  is  discontinuous  at  t  .  We  expand  9(t  e)  in  Taylor 

o  S 

series  about  t  ,  and  we  know  gp  ■  0  on  the  singular  subeurc. 
s 

The  fUrst  non-zero  term  of  the  Taylor  series  contains  the  term 
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Ve  kooir 


So,  (A. 2-6)  turns  out  to  be 


“  It  +  O(e^)  (A.1-8) 


a  ■  -Bgp.  ®  (t„  +  p) 
S 


u^(t)  •  oK(t) 


“  ®  e  -0  “  ® 


t  «  0 . . 


On  the  singular  arc,  the  left  side  can  be  expended  as 


(A.2-9) 


(A.  2-10) 


(A.  2-11) 


(A.  2-12) 


r  i 

<^(■^8  -  e)  -  uCtg  -  e)  =  -  ’’ig^^vtg)]  +  0(£) 


(A.2-13) 


By  using  (A. 2-7)  and  (A. 2-12)  the  rl^t  hand  side  of  (A.2-13)  can 
be  simplified  to 


X*  z* 

rfC(t,  .  e)  -  u(t,  -  s)  .  .0(6) 


We  aubatitute  from  (A.2.13)  Into  (A.2->6)  and  having  K  2q  r  ve 
get 

2a 

(p(tg  +  e)  -  (-1)"*  ^  P(tg)[oK(tg  -  e)  -  u(tg  -  e)]  +  0(5)^ 

(A.2-15) 

From  the  application  of  min  Imam  principle  on  the  non-singular 
subarc  <7 »  1  if  ^(t  +  e)  <  0  and  o  *  -1  if  ^(t  +  e)  >0. 

S  8 

Therefore  ve  have 

(-l)*‘e^(tg)[K(tg  -  e)  iu(tg  -  e)]  <  0  (A.2.16) 

and  from  GilC  condition  ve  have 

(-l)'^(tg)>0  (A.2-17) 

Multiplying  (A,2-l6)  by  (A.2-17)  ve  get 

(-l)^'^V‘^^(t  )[K(t  -  e)  +  u(t  -  e)]  <0  (A.2-18) 

0  s  s 

Since  |u(t)  I  <  K(t)  for  all  t  e  [t^^t^]  and  the  singular  arc  is 

assumed  to  be  interior  almost  everyvhere,  the  bracketed  quantity 

(A.2-18)  regardless  of  +  signs  is  positive,  also  e^*^^(t  )  >  0. 

^  s 

Therefore 

(-1)'^'*’^<0  (A.2-19) 

vhlch  Implies  q+r  is  odd. 


APPENDIX  B 


B.l  Linear  Case 

Assume  to  be  a  nominal  switching  time,  we  try  to  find 

i  ® 

due  to  the  choice  of  t^.  For  this  purpose  we  calculate 


where 


0(tg)  -  Y(t^,t^)§(t^,tQ)  (B.l-3 


For  simplicity  we  drop  tg,t^  and  tQ  ffcm  transition  matrices 


A*(t  )  2:  [^1  ml  -^^t  .  DAt^  (B.1-4 

L-q  V  J  ®  ® 


A+ai  BN-CR“^^1 


Since  x(tQ)  is  fixed  Ax(tQ)  -  0,  and  to  find  A\(tQ),  the 
2d.  X  2ii  matrices  Y,  0,  D,  E  dtb  peurtitioned  to  n  x  n  matrices 
as  the  following. 


Prom  ^3.1-3)  we  can  obtain 


\(tg)  -  !Qc(tg)  (B.1-7) 

where  K  is  em  n  x  n  matrix.  So,  we  can  have 

^x(tg)  «  l:ix(tg)  (B.1-8) 

Prom  (B.1-8)  and  (3.1-17)  is  obtained  as 

Ax(tQ)  .  B  Atg  (B.1-9) 

where 

I  -  [«22  ‘  +^K  .D2^x(tQ)  -D22^(to)] 

(B.l-ID) 


Since  the  inverse  in  (B,l-10)  always  exists,  we  will  have 


X  G 


(B.l-U) 


A\(tp 


(B.1-12) 


vhertt 


L  ■  [Qjj^  *  (’*^11^11  *'*  ■*■  ■*■  '*^1^22^ 

+  S^^x(tJ)  +  ir^(tj)  +  Tj^BK]  (B.1-13) 


M  ■  CQgsF  ”*“  ^'*^21^11  “'’  '*^2^21^*^^0^  *  ^^21®12 

+  E2jX(tJ)  +ff22^(tj)  +'r2jBK  (B.1-14) 


(B.1-15) 


From  (B.1-1),  (B.l-U)  and  (B.1-12)  ve  will  get 


AEEP^  -  (M  -  SL)  1  .  At 


So,  when  At  approaches  zero  we  will  have 


1  •  ^  1 
■  “tg  tg 

___  Step  i 

By  having  TEF  and  for  ea<^  switching  time  tg  we  can 

8 

use  Newton's  method  to  find  tg  at  which  ||IEF||  >  0,  Az(tp  and 

A\(tp  can  he  expressed  in  terms  of  At^  to  the  second  order  in  which 

ease  by  Iterative  methods  we  can  get  the  optimal  t  where 

s 
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||TEPI|  m  0  but  it  will  be  tedious. 


B.2  Hon-Linear  Cause 


In  this  section  we  are  trying  to  find  an  expression  for 
The  solution  to  (3.2.12)  and  (3.2-13)  is 


8*(tp 


s*(to) 


(B.2-1) 


Where 


^f  bt 

(*(t3,to)l-  ^ 


(B.2.2) 


(B.2-3) 


and  the  solutioa  to  (3.2-14-)  and  (3.2-13)  is 


5x(tJ 


8\(tp_ 


[T(t^,t^)) 


5x(0 


(B.2-lv) 


where 


[f(Vt^)] 


Sf  df 


i  i? 


[^(tj.tg)] 


(B.2-5) 
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•  I 


(B.2-6) 


^  order  to  express  &x(tp  and  &\(tp  we  elimijiate  ^(t^) 

Between  (3.2-8)  and  (3.2-10)  and  A\(t„)  between  (3-2-9)  and  (3.2-11) 

8 

we  get 


8x(0 


k*-,)  *  i(<) 

*  uO  J 


At  + 
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Bx(t;) 


(B.a-7) 


From  (B.2-7)  and  (B.2-1)  we  have 


8*(t*) 


8\(t^ 
>  8 


*(tp  +  ictp 

+  \(0  J 


0 

,6\(0. 


O'Ji 


and  from  (B.2-8)  and  (B.2J(-)  we  obtain 


(B.2-8) 


6x(tp 

8\(tp 


[Q<tg)]Atg  +  e(tg) 
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8x(tn) 


(B.2-9) 


where 


a(tg)  - 


i(t;)  +  i(tp' 

k*^  *  ktp 


(B.2-1D) 


6(tg)  •  [F(t^»t^)][^^(tg,tQ)] 


(B.2-11) 
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By  linearizing  (3.1-3)  around  a  nominal  trajectory  we  will  get 

r  <v.  (V.  1  r..,*-;  1  r  “  1 


6x(t;) 


(H(n-l)) 

'  u  'x 


From  (B.2-1)  and  (B.2-12)  we  obtain 

r  0  1 


where 


SKCtfl) 


T(t,)  it. 


T(t,) 


^(n) 
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(H  ) 

• 

• 
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(h"’^) 

^  u  ' 

«t,) 


From  (B.2>13)  and  (B.2-9)  we  have 


[Q(t^)  +  0(tg)7(tg)]Atg  -  u(t^)At 


s'  s 


L  V  J 

From  (3.2-18) 


(B.2.17) 


ATEP-6\(tp  »x(tp  +H.O.T. 


Nov  partitioning  |i(t  ) 
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Vjft,) 
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where  and 


AteP  ■  HiC'tg) 


Stef 

6  t_ 


atep 

stT^ 

8 


(B.2-18) 

(B.2-19) 


Therefore,  similar  to  the  linear  case  we  can  find  an  iterative  rela¬ 
tionship  for  the  switching  time. 
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AFFE»DIX  C 


In  this  appendix  first  we  express  TEP^  as  a  function  of  \(tQ) 


and  tg.  For  this  purpose  we  find  the  final  states  and  costates  of 
the  system  from  (3.1-8)  and  (3.I-I2)  as 


x^(tp 


x(to) 


X(trt) 


O'J 


+  [V(tJ)] 


where  e(t^)  is  defined  in  Appendix  B  and 
8 


ntj) . 


|.V*s) 


and  Vg  are  n  x  1  rectors  as  a  function  of  t^. 


By  definition 


li(tf) -si(tj))V(tp (0.3) 


From  (C.l)  we  hare 


/  x(tj)  .  e^^x(tQ)  *  e^(t^)  * 


I  \(tp  -  ^2 


(C.l) 


(C.2) 


(c.l^) 

(C.5) 


and  fircBa  (C.2)  -  (C.3)  we  will  dbtain 


\  -  +  2Q(tpx"-(tQ)  +  (to)R(tg)\^(tQ)  (C.6) 

Where 


- 

*("2' 

-  ®V1^<®22  - 

se^ 

(C.7) 

8(tJ)  - 

[822  - 

88i2)®(822  - 

sa^] 

(C.8) 

(821- 

S^)X(tg)  + 

(Vg- 

+ 

SV^)] 

(C.9) 

Since  Condition  (3.1-3)  lineer  In  states  and  costates  it  can  be 
expressed  as 

x(tg) 

[V(tb]  ,  -  0  (C.ID) 

where  7(s)  is  an  mx2n  aatrix  is  a  function  of  system  pskrameters. 
From  (C.IO)  and  (3.1-18)  we  will  obtain 

»  T(t^)3  (c.n) 

where  •-  ^ 

tw(ti)]  -  .  .  tv(ti)]t«(ti,t.)i  (C.12) 


(C.13) 


T(t;)  -  [V(tp][T)(t;)] 


and  Wg  are  m  x  n  matrices  and  T  Is  an  m  vector.  So, 


we  will  have 


W2(tJ)).(tj,)  -  T(tJ)  .  «i(tJ)l(t(j) 


(C.U) 


Now  (C.6)  can  he  minimized  with  respect  to  xCt^)  subject  to 
(C.l4)  at  any  t^. 

For  ccB^tatlon  of  [  nl^n  fi]  at  each  t^  we  differentiate 
(C.k)  and  (C.5)  with  respect  to  t 


^(tj  Sei,  ax.(tQ)  dF- 

“St^  -  5^  *(  V  ®12  "S^  ^ 


TT"  V  rr  V  *  ®22  "TTT  sr 


at  each  switching  time  t^  \^( t^) . 


S\^(to) 


and  min  h(t_)  are 
X(tn)  ® 


obtained  so  [min  ^(t^)]  can  be  compxted,  then  by  a  Newton 
8 

iterative  relationship  the  optimal  switching  time  is  reached. 


